Skip to content
  1. Sep 21, 2022
  3. Sep 20, 2022
    • Sergej Sakac's avatar
      BREAKING: Rename Origin (#12258) · e4b6f4a6
      Sergej Sakac authored 
      * BREAKING: Rename Origin

* more renaming

* a bit more renaming

* fix

* more fixing

* fix in frame_support

* even more fixes

* fix

* small fix

* ...

* update .stderr

* docs

* update docs

* update docs

* docs
      e4b6f4a6
  5. Sep 16, 2022
    • Sam Johnson's avatar
      Add special tag to exclude runtime storage items from benchmarking (#12205) · c0e007b5
      Sam Johnson authored 
      * initial setup

* add WhitelistedStorageKeys trait

* add (A, B) tuple implementation for whitelisted_storage_keys()

* fix formatting

* implement WhitelistedStorageKeys for all tuple combinations

* impl_for_tuples up to 128 for WhitelistedStorageKeys

* refactor to #[benchmarking(cached)]

* tweak error message and mark BlockNumber as cached

* add benchmarking(cached) to the other default types

* add docs for benchmarking(cached)

* properly parse storage type declaration

* make storage_alias structs public so we can use them in this macro

* use BTreeMap since TrackedStorageKey missing Ord outside of std

* make WhitelistedStorageKeys accessible

* basic detection of benchmarking(cached) 💥

* proper parsing of #[benchmarking(cached)] from pallet parse macro

* store presence of #[benchmarking(cached)] macro on StorageDef

* will be used for later expansion

* compiling blank impl for WhitelistedStorageKeys

* move impl to expand_pallet_struct

* use frame_support::sp_std::vec::Vec properly

* successfully compiling with storage info loaded into a variable 💥

* plausible implementation for whitelisted_storage_keys()

* depends on the assumption that storage_info.encode() can be loaded
  into TrackedStorageKey::new(..)

* use Pallet::whitelisted_storage_keys() instead of hard-coded list

* AllPallets::whitelisted_storage_keys() properly working 💥

* collect storage names

* whitelisted_storage_keys() impl working 💥



* clean up

* fix compiler error

* just one compiler error

* fix doc compiler error

* use better import path

* fix comment

* whoops

* whoops again

* fix macro import issue

* cargo fmt

* mark example as ignore

* use keyword tokens instead of string parsing

* fix keyword-based parsing of benchmarking(cached)

* preliminary spec for check_whitelist()

* add additional test for benchmarking whitelist

* add TODO note

* remove irrelevant line from example

* use filter_map instead of filter and map

* simplify syntax

Co-authored-by: default avatarKeith Yeung <[email protected]>

* clean up

* fix test

* fix tests

* use keyword parsing instead of string parsing

* use collect() instead of a for loop

Co-authored-by: default avatarKian Paimani <[email protected]>

* fix compiler error

* clean up benchmarking(cached) marking code

* use cloned()

* refactor to not use panic! and remove need for pub types in storage_alias

* remove unneeded use

Co-authored-by: default avatarBastian Köcher <[email protected]>

* remove unneeded visibility changes

* don't manually hard code hash for treasury account as hex

* proper Ord, PartialOrd, and Hash impls for TrackedStorageKey

* now based just on key, and available in no-std

* use BTreeSet instead of BTreeMap

* fix comments

* cargo fmt

* switch to pallet::whitelist and re-do it basti's way :D

* make PartialOrd for TrackedStorageKey consistent with Ord

* more correct implementation of hash-related traits for TrackedStorageKey

* fix integration test

* update TODO

* remove unused keyword

* remove more unused keywords

* use into_iter()

Co-authored-by: default avatarKeith Yeung <[email protected]>

* Update frame/support/procedural/src/pallet/parse/mod.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* add docs for whitelisted

* fix comment

Co-authored-by: default avatarKeith Yeung <[email protected]>
Co-authored-by: default avatarKian Paimani <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
      c0e007b5
  7. Sep 15, 2022
  9. Sep 14, 2022
  11. Sep 13, 2022
    • Keith Yeung's avatar
      Create sp-weights crate to store weight primitives (#12219) · 30e7b1e8
      Keith Yeung authored 
      

* Create sp-weights crate to store weight primitives

* Fix templates

* Fix templates

* Fixes

* Fixes

* cargo fmt

* Fixes

* Fixes

* Use deprecated type alias instead of deprecated unit types

* Use deprecated subtraits instead of deprecated hollow new traits

* Fixes

* Allow deprecation in macro expansion

* Add missing where clause during call macro expansion

* cargo fmt

* Fixes

* cargo fmt

* Fixes

* Fixes

* Fixes

* Fixes

* Move FRAME-specific weight files back to frame_support

* Fixes

* Update frame/support/src/dispatch.rs

Co-authored-by: default avatarKian Paimani <[email protected]>

* Update frame/support/src/dispatch.rs

Co-authored-by: default avatarKian Paimani <[email protected]>

* Update frame/support/src/dispatch.rs

Co-authored-by: default avatarKian Paimani <[email protected]>

* Add missing header

* Rewrite module docs

* Fixes

* Fixes

* Fixes

* Fixes

* cargo fmt

Co-authored-by: default avatarShawn Tabrizi <[email protected]>
Co-authored-by: default avatarKian Paimani <[email protected]>
      30e7b1e8
    • Michal Kucharczyk's avatar
      Remove CanAuthorWith trait (#12213) · 214eb25f
      Michal Kucharczyk authored 
      * Remove CanAuthorWith trait

CanAuthotWith trait removed. Also all dependencies, parameters, type
paramers were removed. This is related to removal of native runtime.

* Remove commented code

* Fix code formatting

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job
      214eb25f
  13. Sep 12, 2022
  15. Sep 08, 2022
  17. Sep 07, 2022
  19. Sep 06, 2022
  21. Sep 05, 2022
    • Davide Galassi's avatar
      Fetch Babe configuration from runtime state (#11760) · 83aec6f9
      Davide Galassi authored 
      * Fetch babe config data from runtime state

* Some renaming

* More renaming

* Final nits

* Fix tests and benches

* Rename  to  in BabeConfiguration

* Remove duplicate babe parameter description

Already specified over the 'PRIMARY_PROBABILITY' constant value

* trigger pipeline

* trigger pipeline
      83aec6f9
  23. Aug 29, 2022
  25. Aug 26, 2022
  27. Aug 25, 2022
  29. Aug 23, 2022
  31. Aug 18, 2022
    • Bastian Köcher's avatar
      Introduce trie level cache and remove state cache (#11407) · 73d9ae32
      Bastian Köcher authored 
      * trie state cache

* Also cache missing access on read.

* fix comp

* bis

* fix

* use has_lru

* remove local storage cache on size 0.

* No cache.

* local cache only

* trie cache and local cache

* storage cache (with local)

* trie cache no local cache

* Add state access benchmark

* Remove warnings etc

* Add trie cache benchmark

* No extra "clone" required

* Change benchmark to use multiple blocks

* Use patches

* Integrate shitty implementation

* More stuff

* Revert "Merge branch 'master' into trie_state_cache"

This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff0364

.

* Improve benchmark

* Adapt to latest changes

* Adapt to changes in trie

* Add a test that uses iterator

* Start fixing it

* Remove obsolete file

* Make it compile

* Start rewriting the trie node cache

* More work on the cache

* More docs and code etc

* Make data cache an optional

* Tests

* Remove debug stuff

* Recorder

* Some docs and a simple test for the recorder

* Compile fixes

* Make it compile

* More fixes

* More fixes

* Fix fix fix

* Make sure cache and recorder work together for basic stuff

* Test that data caching and recording works

* Test `TrieDBMut` with caching

* Try something

* Fixes, fixes, fixes

* Forward the recorder

* Make it compile

* Use recorder in more places

* Switch to new `with_optional_recorder` fn

* Refactor and cleanups

* Move `ProvingBackend` tests

* Simplify

* Move over all functionality to the essence

* Fix compilation

* Implement estimate encoded size for StorageProof

* Start using the `cache` everywhere

* Use the cache everywhere

* Fix compilation

* Fix tests

* Adds `TrieBackendBuilder` and enhances the tests

* Ensure that recorder drain checks that values are found as expected

* Switch over to `TrieBackendBuilder`

* Start fixing the problem with child tries and recording

* Fix recording of child tries

* Make it compile

* Overwrite `storage_hash` in `TrieBackend`

* Add `storage_cache` to  the benchmarks

* Fix `no_std` build

* Speed up cache lookup

* Extend the state access benchmark to also hash a runtime

* Fix build

* Fix compilation

* Rewrite value cache

* Add lru cache

* Ensure that the cache lru works

* Value cache should not be optional

* Add support for keeping the shared node cache in its bounds

* Make the cache configurable

* Check that the cache respects the bounds

* Adds a new test

* Fixes

* Docs and some renamings

* More docs

* Start using the new recorder

* Fix more code

* Take `self` argument

* Remove warnings

* Fix benchmark

* Fix accounting

* Rip off the state cache

* Start fixing fallout after removing the state cache

* Make it compile after trie changes

* Fix test

* Add some logging

* Some docs

* Some fixups and clean ups

* Fix benchmark

* Remove unneeded file

* Use git for patching

* Make CI happy

* Update primitives/trie/Cargo.toml

Co-authored-by: default avatarKoute <[email protected]>

* Update primitives/state-machine/src/trie_backend.rs

Co-authored-by: default avatarcheme <[email protected]>

* Introduce new `AsTrieBackend` trait

* Make the LocalTrieCache not clonable

* Make it work in no_std and add docs

* Remove duplicate dependency

* Switch to ahash for better performance

* Speedup value cache merge

* Output errors on underflow

* Ensure the internal LRU map doesn't grow too much

* Use const fn to calculate the value cache element size

* Remove cache configuration

* Fix

* Clear the cache in between for more testing

* Try to come up with a failing test case

* Make the test fail

* Fix the child trie recording

* Make everything compile after the changes to trie

* Adapt to latest trie-db changes

* Fix on stable

* Update primitives/trie/src/cache.rs

Co-authored-by: default avatarcheme <[email protected]>

* Fix wrong merge

* Docs

* Fix warnings

* Cargo.lock

* Bump pin-project

* Fix warnings

* Switch to released crate version

* More fixes

* Make clippy and rustdocs happy

* More clippy

* Print error when using deprecated `--state-cache-size`

* 🤦



* Fixes

* Fix storage_hash linkings

* Update client/rpc/src/dev/mod.rs

Co-authored-by: default avatarArkadiy Paronyan <[email protected]>

* Review feedback

* encode bound

* Rework the shared value cache

Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.

* Improve of the cache by merging the old and new solution

* FMT

* Please stop coming back all the time :crying:

* Update primitives/trie/src/cache/shared_cache.rs

Co-authored-by: default avatarArkadiy Paronyan <[email protected]>

* Fixes

* Make clippy happy

* Ensure we don't deadlock

* Only use one lock to simplify the code

* Do not depend on `Hasher`

* Fix tests

* FMT

* Clippy 🤦



Co-authored-by: default avatarcheme <[email protected]>
Co-authored-by: default avatarKoute <[email protected]>
Co-authored-by: default avatarArkadiy Paronyan <[email protected]>
      73d9ae32
  33. Aug 15, 2022
  35. Aug 13, 2022
    • Tsvetomir Dimitrov's avatar
      Runtime API versioning (#11779) · 2bff2f84
      Tsvetomir Dimitrov authored 
      * Runtime API versioning

Related to issue #11577

Add support for multiple versions of a Runtime API. The purpose is to
have one main version of the API, which is considered stable and
multiple unstable (aka staging) ones.

How it works
===========
Some methods of the API trait can be tagged with `#[api_version(N)]`
attribute where N is version number bigger than the main one. Let's call
them **staging methods** for brevity.

The implementor of the API decides which version to implement.

Example (from https://github.com/paritytech/substrate/issues/11577#issuecomment-1145347025

):

```
decl_runtime_apis! {
    #{api_version(10)]
    trait Test {
         fn something() -> Vec<u8>;
         #[api_version(11)]
         fn new_cool_function() -> u32;
    }
}
```

```
impl_runtime_apis! {
    #[api_version(11)]
    impl Test for Runtime {
         fn something() -> Vec<u8> { vec![1, 2, 3] }

         fn new_cool_function() -> u32 {
             10
         }
    }
}
```

Version safety checks (currently not implemented)
=================================================
By default in the API trait all staging methods has got default
implementation calling `unimplemented!()`. This is a problem because if
the developer wants to implement version 11 in the example above and
forgets to add `fn new_cool_function()` in `impl_runtime_apis!` the
runtime will crash when the function is executed.

Ideally a compilation error should be generated in such cases.

TODOs
=====

Things not working well at the moment:
[ ] Version safety check
[ ] Integration tests of `primitives/api` are messed up a bit. More
specifically `primitives/api/test/tests/decl_and_impl.rs`
[ ] Integration test covering the new functionality.
[ ] Some duplicated code

* Update primitives/api/proc-macro/src/impl_runtime_apis.rs

Code review feedback and formatting

Co-authored-by: default avatarasynchronous rob <[email protected]>

* Code review feedback

Applying suggestions from @bkchr



* fmt

* Apply suggestions from code review

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Code review feedback

* dummy trait -> versioned trait

* Implement only versioned traits (not compiling)

* Remove native API calls (still not compiling)

* fmt

* Fix compilation

* Comments

* Remove unused code

* Remove native runtime tests

* Remove unused code

* Fix UI tests

* Code review feedback

* Code review feedback

* attribute_names -> common

* Rework `append_api_version`

* Code review feedback

* Apply suggestions from code review

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Code review feedback

* Code review feedback

* Code review feedback

* Use type alias for the default trait - doesn't compile

* Fixes

* Better error for `method_api_ver < trait_api_version`

* fmt

* Rework how we call runtime functions

* Update UI tests

* Fix warnings

* Fix doctests

* Apply suggestions from code review

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Apply suggestions from code review

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Fix formatting and small compilation errors

* Update primitives/api/proc-macro/src/impl_runtime_apis.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

Co-authored-by: default avatarasynchronous rob <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
      2bff2f84
  37. Aug 12, 2022
  39. Aug 11, 2022
  41. Aug 10, 2022
  43. Aug 09, 2022
    • Nazar Mokrynskyi's avatar
      Network sync refactoring (part 6) (#11940) · a685582b
      Nazar Mokrynskyi authored 
      * Extract `NetworkKVProvider` trait in `sc-authority-discovery` and remove unnecessary dependency

* Extract `NetworkSyncForkRequest` trait in `sc-finality-grandpa`

* Relax requirements on `SyncOracle` trait, remove extra native methods from `NetworkService` that are already provided by trait impls

* Move `NetworkSigner` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`

* Move `NetworkKVProvider` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`

* Minimize `sc-authority-discovery` dependency on `sc-network`

* Move `NetworkSyncForkRequest` trait from `sc-finality-grandpa` to `sc-network-common` and de-duplicate methods in `NetworkService`

* Extract `NetworkStatusProvider` trait and de-duplicate methods on `NetworkService`

* Extract `NetworkPeers` trait and de-duplicate methods on `NetworkService`

* Extract `NetworkEventStream` trait and de-duplicate methods on `NetworkService`

* Move more methods from `NetworkService` into `NetworkPeers` trait

* Move `NetworkStateInfo` trait into `sc-network-common`

* Extract `NetworkNotification` trait and de-duplicate methods on `NetworkService`

* Extract `NetworkRequest` trait and de-duplicate methods on `NetworkService`

* Remove `NetworkService::local_peer_id()`, it is already provided by `NetworkStateInfo` impl

* Extract `NetworkTransaction` trait and de-duplicate methods on `NetworkService`

* Extract `NetworkBlock` trait and de-duplicate methods on `NetworkService`

* Remove dependencies on `NetworkService` from most of the methods of `sc-service`

* Address simple review comments
      a685582b
  45. Aug 03, 2022
  47. Aug 02, 2022
  49. Jul 29, 2022
    • Adrian Catangiu's avatar
      Lean BEEFY to Full BEEFY - don't skip (older) mandatory blocks and import justifications (#11821) · 1c6867c6
      Adrian Catangiu authored 
      

* client/beefy: don't accept vote for older rounds

* client/beefy: clean up and reorg the worker struct

* client/beefy: first step towards Full BEEFY

The first step from Lean->Full BEEFY is to have the worker
enforce uninterrupted line of BEEFY finalized mandatory blocks.

There is one mandatory block per session (the first block in the
session). As such, votes processing and votes generation now
enforces that all mandatory blocks are finalized in strict
monotonically increasing sequence and no block 'N' will be worked
on if there is any GRANDPA finalized but BEEFY non-final mandatory
block 'M', where 'M < N'.

Implementation details:

- Introduced 'VoterOracle' to separate the voting decisions logic,
  and track new/pending sessions.

- New sessions get queued up with the worker operating either:
  1. up-to-date - all mandatory blocks leading up to current GRANDPA
     finalized: queue has ONE element, the 'current session' where
     `mandatory_done == true`,
  2. lagging behind GRANDPA: queue has [1, N] elements, where all
     `mandatory_done == false`.
     In this state, everytime a session gets its mandatory block
     BEEFY finalized, the session is popped off the queue,
     eventually getting to operating mode `1. up-to-date`.

- Votes get triaged and those that fall withing the `VoterOracle`
  allowed window get processed, the others get dropped if stale,
  or buffered for later processing (when they reach the window).

- Worker general code was also updated to fall in one of two roles:
  1. react to external events and change internal 'state',
  2. generate events/votes based on internal 'state'.

Signed-off-by: default avataracatangiu <[email protected]>

* client/beefy: sketch idea for block import and sync

Signed-off-by: default avataracatangiu <[email protected]>

* client/beefy: add BEEFY block import

* client/beefy: process justifications from block import

* client/beefy: add TODOs for sync protocol

* client/beefy: add more docs and comments

* client/beefy-rpc: fix RPC error

* client/beefy: verify justification validity on block import

* client/beefy: more tests

* client/beefy: small fixes

- first handle and note the self vote before gossiping it,
- don't shortcircuit on err when processing pending votes.

* client/beefy: remove invalid justifications at block import

* todo: beefy block import tests

* RFC: ideas for multiple justifications per block

* Revert "RFC: ideas for multiple justifications per block"

This reverts commit 8256fb07d3124db69daf252720b3c0208202624d.

* client/beefy: append justif to backend on block import

* client/beefy: groundwork for block import test

* client/beefy: groundwork2 for block import test

* client/beefy: groundwork3 for block import test

* client/beefy: add block import test

* client/beefy: add required trait bounds to block import builder

* remove client from beefy block import, backend gets the job done

Signed-off-by: default avataracatangiu <[email protected]>
      1c6867c6
    • Koute's avatar
      Prevent double allocation of the payload when calling `sp_io::storage::get` (#11523) · c4b607d4
      Koute authored 
      

* Expose allocation stats in `FreeingBumpHeapAllocator`

* Return allocation stats when calling into the runtime

* Bump `parity-scale-codec` to 3.1.3 (fork)

* Prevent double allocation of the payload when calling `sp_io::storage::get`

* Fix tests

* Remove unnecessary `mut`

* Enable the `bytes` feature for `parity-scale-codec` in `sp-runtime-interface`

* Update client/allocator/src/freeing_bump.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Bump `parity-scale-codec` to 3.1.3

* Fix some of the UI tests

Co-authored-by: default avatarBastian Köcher <[email protected]>
      c4b607d4
  51. Jul 26, 2022
    • Sebastian Kunert's avatar
      Prepare for rust 1.62.1 (#11903) · 9f409dc0
      Sebastian Kunert authored 
      

* Update UI test output for rust 1.62.1

* switch ci to staging image to check that everything works

* fix artifacts node-bench-regression-guard

* Imeplement `scale_info::TypeInfo` manually to silence aggressive rust warning

* Fix more clippy lints

* Make clippy happy by relying on auto-deref were possible

* Add tracking issue to the comments

* pin ci image

Co-authored-by: default avataralvicsam <[email protected]>
      9f409dc0
  53. Jul 25, 2022
  55. Jul 18, 2022