* Add ‘transaction_version’ to the signed transaction

This allows hardware wallets to know which transactions they can safely
sign.  To reduce transaction size, I reduced it to a ‘u8’ from a ‘u32’.

Fixes #5951.

* Restore transaction_version to a u32

* Fix comments

`transaction_version` is not part of a tx, but is still signed.

Co-authored-by: Bastian Köcher <[email protected]>

* Fix the test suite

I had forgotten to change the production of transactions in the test
code.

* Fix benchmarks

* Improve docs for `CheckTxVersion` in `frame_system`

Co-authored-by: André Silva <[email protected]>

* Remove spurious cast

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Split operational and normal weight

* Update other parts of the code

* relatively minor changes to add/fix behavior

* Limit reported block weight for fee adjustment

* fix test

* explicit match

* Move common logic out

* Update frame/system/src/lib.rs

Co-authored-by: Gavin Wood <[email protected]>

* API improvement @bkchr



Co-authored-by: Gavin Wood <[email protected]>

* Minor update on example-offchain-worker code and comment

Signed-off-by: Jimmy Chu <[email protected]>

* Update frame/system/src/offchain.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update system weights

* Use maximum block weight for scheduler

* Update offences to use full block

* Move weight inside if statement

* Add one read to offences `on_initialize`

* Delete factory test

* Revert "Delete factory test"

This reverts commit 8f95aacd63a028ef1b415185b45367b4140d86fd.

* Revert "Add one read to offences `on_initialize`"

This reverts commit 7df7ebc73625ed79b14086f13c247d4058ee87d6.

* Revert "Move weight inside if statement"

This reverts commit 87277d07913a7d1868eeee85ef4673f51ee4013b.

* Revert "Update offences to use full block"

This reverts commit 0bbe0ce18e9419b032157f7d37dea6481078cdc0.

* Use scheduler in Sudo

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <[email protected]>

* Revert "Use scheduler in Sudo"

This reverts commit 95bd2768dfea100bdf682cf4fe6c0f46e8e1f66e.

* remove max extrinsic weight (it does nothing useful)

* fix tests

* introduce `sudo_unchecked_weight`

* bump spec version

* scheduler 80 percent of maximum

* Update `set_changes_trie_config` weight

* Update bin/node/runtime/src/lib.rs

* Update frame/democracy/src/tests.rs

* Update tests.rs

* update based on feedback

Co-authored-by: Alexander Popiak <[email protected]>

* Make `Digest` support `StorageAppend`

This adds support for `StorageAppend` to `Digest`. Digest is just a
wrapper around a `Vec` and we abuse the fact that SCALE does not puts
any special marker into the encoding for structs. So, we can just append
to the encoded Digest. A test is added that ensures, if the `Digest`
format ever changes, we remove this optimization.

* Update weight

* Update frame/support/src/storage/mod.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/system/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

Co-authored-by: Alexander Popiak <[email protected]>

* Start improving `storage_append`

* Fix some stuff

* Fix compilation

* Update docs and add new test

* More tests

* Test kill + append

* Frame System Benchmarking

* Add to substrate node, avoid divide by zero errors in analysis

* reduce features

* some fixes

* copy pasta

* use append api to deposit events

* use optimized append

* one more optimization

* fix bug

* fix issues

* address review

* Introduce `BlockExectionWeight` and `ExtrinsicBaseWeight`

* Add new traits everywhere

* Missed one update

* fix tests

* Update `check_weight` logic

* introduce `max_extrinsic_weight` function

* fix + add tests

* format nits

* remove println

* make test a bit more clear

* Remove minimum weight

* newlines left over from find/replace

* Fix test, improve clarity

* Fix executor tests

* Extrinsic base weight same as old `MINIMUM_WEIGHT`

* fix example test

* Expose constants

* Add test for full block with operational and normal

* Initiate test environment with `BlockExecutionWeight` weight

* format nit

* Update frame/system/src/lib.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Replace `TransactionBaseFee` with `ExtrinsicBaseWeight` (#5761)

* Replace `TransactionBaseFee` with `ExtrinsicBaseFee`

* Fix stuff

* Fix and make tests better

* Forgot to update this test

* Fix priority number in test

* Remove minimum weight from merge

* Fix weight in contracts

* remove `TransactionBaseFee` from contract tests

* Let `register_extra_weight_unchecked` go past `MaximumBlockWeight`

* address feedback

Co-authored-by: Kian Paimani <[email protected]>

Co-Authored-By: thiolliere <[email protected]>

* enum Pays for PaysFee

* Fix doc test

* Update bin/node/executor/tests/basic.rs

* Update bin/node/executor/tests/basic.rs

* Migrate away from SimpleDispatchInfo

* Fix imports

* Better doc

* Update lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* New approach to offchain signing.

* Use in im-online

* Rewrite to use Account<T>

* DRY signing.

* Implement send_raw_unsigned_transaction

* WiP

* Expunge LocalCall

* Expunge LocalCall

* Fix compilation.

* Solve call.

* Make it compile again.

* Finalize implementation.

* Change CreateTransaction

* Clear CreateTransaction.

* Add price payload

* Send raw transaction

* Submit signed payload / unsigned transaction (WIP)

* Supertrait requirements on T::Signature

* Validate signature of payload on an unsigned transaction

* Fix encoding - part 1

* Make it compile.

* Fix compilation of unsigned validator.

* Pass price payload to the transaction

* Make block number part of the signed payload

* Send signed transaction

* Implement all_accounts, any_account

* Fix formatting

* Implement submit_transaction

* Submit signed transaction (ForAll, ForAny)

* Fix formatting

* Implement CreateSignedTransaction

* Move sign and verify to AppCrypto

* Sign transaction

* Call `use_encoded`

* Remove SubmitAndSignTransaction

* Implement runtime using new SigningTypes

* Adapt offchain example to changes

* Fix im-online pallet

* Quick fix: rename AuthorityId2

* Fix offchain example tests

* Add a comment on why keystore is required in unsigned transaction test

* Use UintAuthorityId instead of u64

* WIP

* Remove IdentifyAccount from UintAuthorityId

* Implement PublicWrapper type

* Fix im-online tests

* Fix runtime test

* Bump spec version

* Fix executor tests

* Rename ImOnlineAuthId -> ImOnlineAuthorityId and formatting

* Fix merge

* Documentation

* Revert u64 -> UintAuthorityId conversion

* Fix string errors

* Document public members in offchain module

* Introduce SubmitTransaction

* Update pallets to use SubmitTransaction

* WIP

* Use SubmitTransaction in offchain

* Use `submit_unsigned_transaction`

* Fix tests

* Update docs

* Remove SigningTypes requirement from `SendTransactionTypes`

* Fix tests

* Update frame/system/src/offchain.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Update frame/system/src/offchain.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Update frame/example-offchain-worker/src/tests.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Update frame/system/src/offchain.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Update frame/system/src/offchain.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Remove leftover from previous iterations

* Change enum to struct

* Remove public

* Move mock to node/executor/tests

* Cleanup test-helpers

* Make `application-crypto` `std` feature internal

The macros should not generate code that requires that the calling crate
has a feature with the name `std` defined.

* Revert cargo lock update

* Use TestAuthorityId from common

* Restore members of account to public

* Tidy up imports

* Fix benchmarking pallet

* Add tests demonstrating ForAll, ForAny on signer

* Move definition of AppCrypto

in example-offchain-worker
from tests to mod::crypto

* Cleanup stray comment

* Fix ValidTransaction

* Re-fix CreateSignedTransaction

* Address PR feedback

* Add can_sign method to signer

* Propagate error

* Improve documentation

* Fix vec! macro not available

* Document SendTransactiontypes

* Add some docs.

* Split signing examples

* Add tests for signing examples

* WIP can_sign - PR feedback

* WIP

* Split for_any / for_all into different calls

* Verify payload and signature in test

* Fix can_sign implementation

* Fix impl_version

* Import Box from sp_std

* Create issues for TODOs

* Ignore doctest.

* Add test directly to system. Adjust UintTypes.

* Add some tests to account filtering.

* Remove code samples and point to example offchain worker

* Fix doc links

* Fix im-online tests using signatures.

Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Co-Authored-By: thiolliere <[email protected]>

* Add transaction_version

* Semantic versioning for runtimes

* Move new field to bottom

* Versioning

* Runtime versioning stuff.

* Fix test

* Adds tests and fixes bugs

* Bump runtime

Co-authored-by: Bastian Köcher <[email protected]>

Co-Authored-By: Kian Paimani <[email protected]>

* bumping version

* cargo update

* adding changelog

* Add tests for weight refund

* Update frame/system/src/lib.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Fixed formatting

* Format fixes

Co-authored-by: Kian Paimani <[email protected]>

This refunds weight and the weight bases fee back
to the sender of an extrinsic after the dispatch.

* Refactor SignedExtension

* Move DispatchInfo Associated type to Dispatchable
* Bound Call: Dispatchable
* Pass PostDispatchInfo to post_dispatch
* Pass DispatchInfo by reference to avoid clones

* Whitespace fix

Co-Authored-By: Tomasz Drwięga <[email protected]>

* Style changes from code review

Co-Authored-By: Kian Paimani <[email protected]>

* Only decalre in test mod to remove warning

* Deduplicate Call definition

* Bound frame_system::trait::Call by Dispatchable

* Introduce DispatchInfoOf type alias

* Whitespace fix from review

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Update to latest staking

* generate tests for benchmarking

* add tests, fix warnings

* starting on democracy

* impl_benchmark_tests

* Way more readable

* add test feature flag (does this work?)

* Fix `successful_origin` impl

* democracry benchmark tests

* Fix example benchmarks, add tests

* identity benchmark tests

* Update im-online benchmark tests

* try to add session benchmarking tests (problem with mock)

* staking and timestamp

* add test for treasury, issue with dynamic contains

* utility

* Vesting

* test instead of check

* hide until we figure out what is wrong

* add docs

* close code

* Create custom mock for session-pallet-benchmarking

* Use refcell pattern

* make un-pub

* test-linux-stable includes `runtime-benchmarks` feature

* Revert "test-linux-stable includes `runtime-benchmarks` feature"

This reverts commit a2dab38abd18ac3eb8a6220e4a00e687740bd38c.

* run tests in `--release`

* undo balance change

* build wasm

* Mandatory dispatch class

* Tweaks

* Docs

* Fix test

* Update frame/support/src/weights.rs

Co-Authored-By: joe petrowski <[email protected]>

* Introduce logic that was stated in PR.

* Use

* Docs.

* Fix test

* Fix merge

* Update frame/support/src/weights.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Fix.

* Fix

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* EnsureOrigin is a frame abstraction - it should be in frame_support

* Fixes

* Don't populate runtime events in genesis

* typo

* Change to block zero

* Fix vesting tests

* Update frame/system/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update frame/system/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Add test

* Fix test

* Fix contract tests

* Fix phragmen tests

* Fix Generic Assets Tests

* Fix offences tests

* Fix im-online

* fix recovery

* Fix utility tests

* Shorter

* Use ext

Co-authored-by: Bastian Köcher <[email protected]>

* Add docs.rs metadata to all cargo.toml files

* Remove docs.rs metada in substrate's cargo.toml

* Initial skeleton for offchain phragmen

* Basic compact encoding decoding for results

* add compact files

* Bring back Self::ensure_storage_upgraded();

* Make staking use compact stuff.

* First seemingly working version of reduce, full of todos

* Everything phragmen related works again.

* Signing made easier, still issues.

* Signing from offchain compile fine 😎

* make compact work with staked asssignment

* Evaluation basics are in place.

* Move reduce into crate. Document stuff

* move reduce into no_std

* Add files

* Remove other std deps. Runtime compiles

* Seemingly it is al stable; cycle implemented but not integrated.

* Add fuzzing code.

* Cleanup reduce a bit more.

* a metric ton of tests for staking; wip 🔨

* Implement a lot more of the tests.

* wip getting the unsigned stuff to work

* A bit gleanup for unsigned debug

* Clean and finalize compact code.

* Document reduce.

* Still problems with signing

* We officaly duct taped the transaction submission stuff. 🤓

* Deadlock with keys again

* Runtime builds

* Unsigned test works 🙌

* Some cleanups

* Make all the tests compile and stuff

* Minor cleanup

* fix more merge stuff

* Most tests work again.

* a very nasty bug in reduce

* Fix all integrations

* Fix more todos

* Revamp everything and everything

* Remove bogus test

* Some review grumbles.

* Some fixes

* Fix doc test

* loop for submission

* Fix cli, keyring etc.

* some cleanup

* Fix staking tests again

* fix per-things; bring patches from benchmarking

* better score prediction

* Add fuzzer, more patches.

* Some fixes

* More docs

* Remove unused generics

* Remove max-nominator footgun

* Better fuzzer

* Disable it ❌



* Bump.

* Another round of self-review

* Refactor a lot

* More major fixes in perThing

* Add new fuzz file

* Update lock

* fix fuzzing code.

* Fix nominator retain test

* Add slashing check

* Update frame/staking/src/tests.rs

Co-Authored-By: Joshy Orndorff <[email protected]>

* Some formatting nits

* Review comments.

* Fix cargo file

* Almost all tests work again

* Update frame/staking/src/tests.rs

Co-Authored-By: thiolliere <[email protected]>

* Fix review comments

* More review stuff

* Some nits

* Fix new staking / session / babe relation

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* Update primitives/phragmen/compact/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* Some doc updates to slashing

* Fix derive

* Remove imports

* Remove unimplemented tests

* nits

* Remove dbg

* Better fuzzing params

* Remove unused pref map

* Deferred Slashing/Offence for offchain Phragmen  (#5151)

* Some boilerplate

* Add test

* One more test

* Review comments

* Fix build

* review comments

* fix more

* fix build

* Some cleanups and self-reviews

* More minor self reviews

* Final nits

* Some merge fixes.

* opt comment

* Fix build

* Fix build again.

* Update frame/staking/fuzz/fuzz_targets/submit_solution.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update frame/staking/src/slashing.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update frame/staking/src/offchain_election.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Fix review comments

* fix test

* === 🔑

 Revamp without staking key.

* final round of changes.

* Fix cargo-deny

* Update frame/staking/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: thiolliere <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

* Add origin bounds to benchmark macro.

* Add democracy benchmark.

* Fix tests

* Remove collective from frame/benchmarking, partially use EnsureOrigin.

* Remove collective stuff.

* Make previous benches compile again.

* Remove comments.

* Make prev bench to work again.

* Add remove votes.

* Add new proxy calls.

* Add runtime-benchmarks guard to EnsureOrigin and implementations.

* Refactor.

* Add missing import.

* Remove duplicated import

* Fix features.

* Add some missing features.

* Update frame/collective/Cargo.toml

Co-Authored-By: Kian Paimani <[email protected]>

* Update frame/democracy/src/benchmarking.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Update frame/democracy/src/benchmarking.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Add referendums to state.

* populate vecs before call

* Update weight docs

* More fixes and weight docs

* More updates

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Disallow default weight

* Fix build and test

* Fix tests

* Fix another beloved ui test.

* fix beloved trybuild tests

* fix treasury?

* Final test fix

* Fix build

* Fix another one

* Fix

* More doctest fix

* bump pre-version

* Changelog alpha.4->alpha.5

* repo + homepage to sp-consensus-vrf

* Add default rocksdb feature to `frame-benchmarking-cli` (#5367)

* Add default rocksdb feature

* Update Cargo.toml

Co-authored-by: Shawn Tabrizi <[email protected]>

* Adds new event phase `Initialization`

Every event that was deposited inside of `on_initialize` was assigned to
the `ApplyExtrinsic(0)` phase which wasn't correct. This pr introduces a
new phase `Initialization`. This is the new phase while initializing
a block. After initialization we switch to `ApplyExtrinsic(N)` and at
the end to `Finalization` as before.

* Set `ExecutionPhase` in `initialize`

* Increment `spec_version`

* Factor out can_set_code

The motivation for this feature is parachain runtime upgrades, which
happen in two different moments: the initial call, when the checks
should be performed, and at the actual upgrade block. It's much
better and more maintainable to just call `can_set_code` in the
function than to manually copy those checks.

I'm not entirely thrilled with the interface, which requires
cloning the entire code vector in `set_code`. However, it looks
like something in `decl_module!` does not play nicely with
references in its function arguments. If `code` has a lifetime,
it _must_ be named, which gives us this signature.

```rust
pub fn can_set_code<'a>(origin, code: &'a [u8]) {
```

Unfortunately, attempting to compile with that signature generates
a very large and baffling collection of build errors, so I decided
to abandon that path for now.

* make can_set_code non-dispatchable per PR revew

Not only can we now borrow the `code` slice, but we also no longer
need to worry about people sending a `can_set_code` transaction because
of some misunderstanding.

* move into existing impl block

* Remove migration code.

* Fix cargo

* Bump spec version

* Version bump

* Split generate_changelog.sh into separate script

Can be run in the format `generate_changelog.sh $previous_version $version`.

* remove early exit from publish_draft_release.sh

* adding changelog

* ci: change last_github_release to also find pre-releases

Co-authored-by: Martin Pugh <[email protected]>