Skip to content
  1. Jan 23, 2023
    • Sam Johnson's avatar
      new proc-macro-based benchmarking syntax (#12924) · 42e5c27c
      Sam Johnson authored 
      * add stub for new benchmark macro

* benchmark syntax

* add #[extrinsic call] separator

* parse #[benchmark] item as a function

* proper emission of error when #[extrinsic_call] annotation is missing

* clean up

* enclosing module via benchmarks! { } working

* use an attribute macro on the module instead of benchmarks! { }

* cargo fmt

* working component implementation

* WIP

* working

* add syntax for Linear<A, B>

* parsing of param ranges (still need to build tuple though)

* params parsing WIP

* clean up (don't need extrinsic call name)

* use proper Result syntax for BenchmarkDef parsing

* proper parsing of Linear<0, 1> style args

* successfully parse and make use of linear component ranges 💥

* rename support variable => home because eventually will be moved

* compile-time check that param range types implement ParamRange

* switch to using balances as example, failing on instance pallet

* successfully set up __origin and __call with balances 💥

* clean up

* use a module

* don't need a variable for transfer

* rename benchmark_transfer -> transfer because no longer conflicts

* clean up

* working with transfer_increasing_users as well 💥

* re-add BareBlock

* add comments for undocumented structs+functions+traits

* refactor in preparation for removing module requirements

* switch to a block instead of a module

* use the outer macro pattern to to enable #[benchmarks] aggregation

* successfully generate SelectedBenchmark 💥

* implement components for SelectedBenchmark

* implement instance for SelectedBenchmark

* properly track #[extra]

* working impl for fn benchmarks()

* run_benchmarks WIP

* finish run_benchmark! impl 💥

* import balances transfer_best_case benchmark

* import transfer_keep_alive balances pallet benchmark

* import set_balance_creating balances pallet benchmark

* import set_balance_killing balances pallet benchmark

* import force_transfer balances pallet benchmark

* add #[extra] annotation and docs to transfer_increasing_users

* import transfer_all balances pallet benchmark

* import force_unreserve balances pallet benchmark

* prepare to implement impl_benchmark_test_suite!

* ensure tests cover #[extra] before and after #[benchmark] tag

* refactor

* clean up

* fix

* move to outer

* switch to benchmarks/instance_benchmarks

* test impl almost done, strange compiler error

* benchmark test suites working 💥

* clean up

* add stub and basic parsing for where_clause

* working except where clause and extrinsic calls containing method chains

* assume option (2) for now wrt https://github.com/paritytech/substrate/pull/12924#issuecomment-1372938718

* clean up

* switch to attribute-style

* properly handle where clauses

* fix subtle missing where clause, now just MessageQueue issues

* fix block formatting in message-queue pallet

* switch to block vs non-block parsing of extrinsic call

* working now but some benchmark tests failing

* message-queue tests working (run order issue fixed) 🎉



* add comments and internal docs for fame_support_procedural::benchmark

* fix license years

* docs for lib.rs

* add docs to new support procedural macros

* don't allow #[benchmark] outside of benchmarking module

* add docs

* use benchmark(extra, skip_meta) style args

* update docs accordingly

* appease clippy

* bump ci

* add notes about `extra` and `skip_meta`

* fix doc tests

* re-run CI

* use `ignore` instead of `no_run` on doc examples

* bump CI

* replace some if-lets with if-elses

* more refactoring of if-let statements

* fix remaining if-lets in BenchmarkDef::from()

* fix if-lets in benchmarks()

* fix remaining if-lets, use nested find_map for extrinsic call

* switch to use #[extrinsic_call] or #[block] situationally

* refactor ExtrinsicCallDef => BenchmarkCallDef

* update docs with info about #[block]

* add macro stub for #[extrinsic_call]

* fix docs and add stub for #[block] as well

* remove unused extern crate line

* fix clippy nits

* Use V2 bench syntax in pallet-example-basic

Just testing the dev-ex...

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>

* carry over comment

* use curly-brace style for impl_benchmark_test_suite!

* remove unneeded parenthesis

* proper handling of _() extrinsic call style

* add docs for _() syntax

* fix crate access

* simplify keyword access

Co-authored-by: default avatarKeith Yeung <[email protected]>

* simplify module content destructuring

Co-authored-by: default avatarKeith Yeung <[email protected]>

* fix crate access "frame_benchmarking" => "frame-benchmarking", compiles

* use _() extrinsic call syntax where possible in balances

* simplify attr.path.segments.last()

Co-authored-by: default avatarKeith Yeung <[email protected]>

* fix compile error being suppressed

* simplify extrinsic call keyword parsing

Co-authored-by: default avatarKeith Yeung <[email protected]>

* use ? operator instead of return None

Co-authored-by: default avatarKeith Yeung <[email protected]>

* rename generics => type_use_generics
rename full_generics => type_impl_generics

* simplify extrinsic call extraction with transpose

* bump CI

* nit

* proper handling of too many + too few block/extrinsic call annotations

* change to B >= A

Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>

* remove unneeded ignore

Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>

* remove another ignore

Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>

* add ui tests

* use _() style extrinsic call on accumulate_dummy

Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>

* add range check to ParamRange

* ui test for bad param ranges

* fix failing example

* add ignore back to other failing example

* tweak expr_call span

Co-authored-by: default avatarKeith Yeung <[email protected]>

* fix typo

* eliminate a match

Co-authored-by: default avatarKeith Yeung <[email protected]>

* change pub fn benchmarks to return Result<TokenStream>

* fix origin error span

* more informative error for invalid benchmark parameter name

* fix spans on a few benchmark errors

* remove unneeded clone

* refactor inner loop of benchmark function parsing

* preserve mod attributes

* refactor outer loop of benchmark def parsing code, greatly simplified

* simplify to use a ? operator when parsing benchmark attr path

* fix another ? operator

* further simplify benchmark function attr parsing with more ? ops

* refactor extrinsic call handling to use if let rather than match

* replace is_ok => is_err

Co-authored-by: default avatarKeith Yeung <[email protected]>

* re-use name during expansion of benchmark def

* remove unneeded clone

* fix span for origin missing error

* fix missing semi

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarKeith Yeung <[email protected]>
Co-authored-by: parity-processbot <>
      42e5c27c
  3. Jan 22, 2023
  5. Jan 21, 2023
  7. Jan 20, 2023
  9. Jan 19, 2023
    • Michal Kucharczyk's avatar
      txpool: LOG_TARGET const added (#13180) · 57b1de3f
      Michal Kucharczyk authored 
      

* txpool: LOG_TARGET const added

part of: #12873

* LOG_TARGET added to tests mod

* txpool::api for api

* Apply suggestions from code review

Co-authored-by: default avatarBastian Köcher <[email protected]>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: default avatarBastian Köcher <[email protected]>
Co-authored-by: command-bot <>
      57b1de3f
    • Alexander Theißen's avatar
      Remove fixtures from crate (#13181) · cce6d406
      Alexander Theißen authored
      cce6d406
    • Gavin Wood's avatar
      Make DispatchError impl MEL (#13169) · 2d7fa18e
      Gavin Wood authored 
      

* Make DispatchError impl MEL

* Upgrade SCALE codec to support `codec(skip)` for MEL

Co-authored-by: default avatarBastian Köcher <[email protected]>
      2d7fa18e
    • Sebastian Kunert's avatar
      Notification-based block pinning (#13157) · 9be65570
      Sebastian Kunert authored 
      

* Worker

* Reorganize and unpin onnotification drop

* Pin in state-db, pass block number

* Pin blocks in blockchain db

* Switch to reference counted LRU

* Disable pinning when we keep all blocks

* Fix pinning hint for state-db

* Remove pinning from backend layer

* Improve readability

* Add justifications to test

* Fix justification behaviour

* Remove debug prints

* Convert channels to tracing_unbounded

* Add comments to the test

* Documentation and Cleanup

* Move task start to client

* Simplify cache

* Improve test, remove unwanted log

* Add tracing logs, remove expect for block number

* Cleanup

* Add conversion method for unpin handle to Finalitynotification

* Revert unwanted changes

* Improve naming

* Make clippy happy

* Fix docs

Co-authored-by: default avatarMichal Kucharczyk <[email protected]>

* Use `NumberFor` instead of u64 in API

* Hand over weak reference to unpin worker task

* Unwanted

* &Hash -> Hash

* Remove number from interface, rename `_unpin_handle`, LOG_TARGET

* Move RwLock one layer up

* Apply code style suggestions

* Improve comments

* Replace lru crate by schnellru

* Only insert values for pinned items + better docs

* Apply suggestions from code review

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Improve comments, log target and test

Co-authored-by: default avatarMichal Kucharczyk <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
      9be65570
    • Tsvetomir Dimitrov's avatar
      Expose `UnknownBlock` error via `ApiError` (#12707) · 7a1958ca
      Tsvetomir Dimitrov authored 
      

* Expose `UnknownBlock` error via `ApiError`

In [certain cases](https://github.com/paritytech/polkadot/issues/5885) a
runtime api is called for an unknown block. For example a block which is
already pruned or on an abandon fork.

In such cases the correct error is returned but it is wrapped in
`ApiError::Application` and the only way to figure out what is the
problem is to inspect the actual message in the error. In polkadot for
example this usually happens when the runtime api version is being
queried. It's beneficial to be able to clearly separate such errors so i
that when they occur the client side can handle them more gracefully.
E.g. log less stressful error message than `State already discarded for
BlockId` or cancel any pending work related on this block.

* Update primitives/api/src/lib.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

Co-authored-by: default avatarBastian Köcher <[email protected]>
      7a1958ca
  11. Jan 18, 2023
  13. Jan 17, 2023
  15. Jan 16, 2023
    • Michal Kucharczyk's avatar
      txpool: don't maintain the pool during major sync (#13004) · 5134dabb
      Michal Kucharczyk authored 
      

* txpool: don't maintain the pool during major sync

Fix shall prevent from wasting the CPU during the major sync. No actions
are actually required in transaction pool during the major sync.

Fixes: #12903

* passing sync_oracle to maintain method

* fixed: builder, txpool tests

* do not maintain tx-pool if node gone out of sync

* EnactmentAction: all logic moved to EnactmentState

Tests to be done.

* maintain guard logic moved directly to MaintainedTransactionPool

* minor fixes

* EnactmentAction: all logic moved to EnactmentState (again)

* SyncOracle fixes here and there

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* sync_oracle removed

* spelling + fmt + doc

* Review suggestions applied

* log::info -> debug

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: parity-processbot <>
Co-authored-by: default avatarBastian Köcher <[email protected]>
      5134dabb
    • JP's avatar
      More improvements for the crate publishing pipeline (#13153) · 21141f5d
      JP authored 
      * more improvements for the crate publishing pipeline

* move default definitions to the publish-crates script

* add script to check the crate publishing pipeline at the start

* fix yaml references

* move more variables to .crates-publishing-pipeline

* separate .crates-publishing-pipeline from .crates-publishing-variables

* clean up redundant and unused code
      21141f5d
    • Bastian Köcher's avatar
      pallet-offences-benchmarking: Box events in verify (#13151) · 52a56e5f
      Bastian Köcher authored 
      * pallet-offences-benchmarking: Box events in verify

Events in frame are represented by an enum in the pallet and the runtime. The size of an enum in
Rust depends on the size of biggest variant. This means we always need to allocate memory for the
biggest variant when allocating memory for an event. The offences benchmarking is verifying the
benchmarking results by checking the events. To check the events it is generating all the expected
events. With the recent changes in Polkadot the events are too big and lead to issues when running
this verify functions. The solution is to box each event, as the vector holding all the events will
then only need to hold fat pointers * expected events, instead of size_of(event) * expected events.
This issue isn't a problem in production, as we never read the events on chain. When we are reading
the events, it is done in an offchain context and they are only decoded one by one.

Besides that this also enables the benchmarking verification for everyone running these benchmarks.

* FMT

* Disable checking again
      52a56e5f
  17. Jan 15, 2023
  19. Jan 13, 2023
  21. Jan 12, 2023
  23. Jan 11, 2023
  25. Jan 10, 2023