* Support multiple, mergable vesting schedules

* Update node runtime

* Remove some TODO design questions and put them as commennts

* Update frame/vesting/src/benchmarking.rs

* Syntax and comment clean up

* Create filter enum for removing schedules

* Dry vesting calls with do_vest

* Improve old benchmarks to account for max schedules

* Update WeightInfo trait and make dummy fns

* Add merge_schedule weights

* Explicitly test multiple vesting scheudles

* Make new vesting tests more more clear

* Apply suggestions from code review

* Update remove_vesting_schedule to error with no index

* Try reduce spacing diff

* Apply suggestions from code review

* Use get on vesting for bounds check; check origin first

* No filter tuple; various simplifications

* unwrap or default when getting user schedules

* spaces be gone

* ReadMe fixes

* Update frame/vesting/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* address some comments for docs

* merge sched docs

* Apply suggestions from code review

Co-authored-by: Shawn Tabrizi <[email protected]>

* log error when trying to push to vesting vec

* use let Some, not is_some

* remove_vesting_schedule u32, not optin

* new not try_new, create validate builder; VestingInfo

* Merge prep: break out tests and mock

* Add files forgot to include in merge

* revert some accidental changes to merged files

* Revert remaining accidental file changes

* More revert of accidental file change

* Try to reduce diff on tests

* namespace Vesting; check key when key should not exist;

* ending_block throws error on per_block of 0

* Try improve merge vesting info comment

* Update frame/vesting/src/lib.rs

Co-authored-by: Kian Paimani <[email protected]>

* add validate + correct; handle duration > blocknumber

* Move vesting_info module to its own file

* Seperate Vesting/locks updates from writing

* Add can_add_vesting schedule

* Adjust min vested transfer to be greater than all ED

* Initial integrity test impl

* merge_finished_and_yet_to_be_started_schedules

* Make sure to assert storage items are cleaned up

* Migration initial impl (not tested)

* Correct try-runtime hooks

* Apply suggestions from code review

Co-authored-by: Shawn Tabrizi <[email protected]>

* header

* WIP: improve benchmarks

* Benchmarking working

* benchmarking: step over max schedules

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Simplify APIs by accepting vec; convert to bounded on write

* Test:  build_genesis_has_storage_version_v1

* Test more error cases

* Hack to get polkadot weights to work; should revert later

* Improve benchmarking; works on polkadot

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* WIP override storage

* Set storage not working example

* Remove unused tests

* VestingInfo: make public, derive MaxEndcodedLen

* Rename ending_block to ending_block_as_balance

* Superificial improvements

* Check for end block infinite, not just duration

* More superficial update

* Update tests

* Test vest with multi schedule

* Don't use half max balance in benchmarks

* Use debug_assert when locked is unexpected 0

* Implement exec_action

* Simplify per_block calc in vesting_info

* VestingInfo.validate in add_vesting_schedule & can_add_vesting_schedule

* Simplify post migrate check

* Remove merge event

* Minor benchmarking updates

* Remove VestingInfo.correct

* per_block accesor max with 1

* Improve comment

* Remoe debug

* Fix add schedule comment

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* no ref for should_remove param

* Remove unused vestingaction derive

* Asserts to show balance unlock in merge benchmark

* Remove unused imports

* trivial

* Fix benchmark asserts to handle non-multiple of 20 locked

* Add generate_storage_info

* migration :facepalm

* Remove per_block 0 logic

* Update frame/vesting/src/lib.rs

* Do not check for ending later than greatest block

* Apply suggestions from code review

* Benchmarks: simplify vesting schedule creation

* Add log back for migration

* Add note in ext docs explaining that all schedules will vest

* Make integrity test work

* Improve integrity test

* Remove unnescary type param from VestingInfo::new

* Remove unnescary resut for ending_block_as_balance

* Remove T param from ending_block_as_balance

* Reduce visibility of raw_per_block

* Remove unused type param for validate

* update old comment

* Make log a dep; log warn in migrate

* VestingInfo.validate returns Err(()), no T type param

* Try improve report_schedule_updates

* is_valid, not validate

* revert node runtime reorg;

* change schedule validity check to just warning

* Simplify merge_vesting_info return type

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Kian Paimani <[email protected]>

* Add warning for migration

* Fix indentation

* Delete duplicate warnings

* Reduce diff in node runtime

* Fix benchmark build

* Upgrade cargo.toml to use 4.0.0-dev

* Cleanup

* MaxVestingSchedulesGetter initial impl

* MinVestedTransfer getter inintial impl

* Test MaxVestingSchedules & MinVestedTransfer getters; use getters in benchmarks

* Run cargo fmt

* Revert MinVestedTransfer & MaxVestingSchedules getters; Add integrity test

* Make MAX_VESTING_SCHEDULES a const

* fmt

* WIP: benchmark improvements

* Finish benchmark update

* Add test for transfer to account with less than ed

* Rm min_new_account_transfer; move sp-io to dev-dep

* Reduce cargo.toml diff

* Explain MAX_VESTING_SCHEDULES choice

* Fix after merge

* Try fix CI complaints

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fmt

* trigger

* fmt

Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: kianenigma <[email protected]>

* continue in pow loop instead of returning

* change other return statements, add logs

* remove redundant logs

* formatting fix

* remove log

* Add some commas to make rustfmt happy

Co-authored-by: Wei Tang <[email protected]>
Co-authored-by: Wei Tang <[email protected]>

* Display warning on try-runtime version mismatch

* detect spec-name mismatch in try-runtime

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update utils/frame/remote-externalities/src/rpc_api.rs

Co-authored-by: Chevdor <[email protected]>

* remove unused import

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Chevdor <[email protected]>

Bumps [directories](https://github.com/soc/directories-rs) from 3.0.1 to 3.0.2.
- [Release notes](https://github.com/soc/directories-rs/releases)
- [Commits](https://github.com/soc/directories-rs/commits

)

---
updated-dependencies:
- dependency-name: directories
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [prost](https://github.com/tokio-rs/prost) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.7.0...v0.8.0

)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [bitflags](https://github.com/bitflags/bitflags) from 1.2.1 to 1.3.2.
- [Release notes](https://github.com/bitflags/bitflags/releases)
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md)
- [Commits](https://github.com/bitflags/bitflags/compare/1.2.1...1.3.2

)

---
updated-dependencies:
- dependency-name: bitflags
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add comments and refactor Sandbox module

* Adds some comments

* Add wasmtime instance to the sandbox and delegate calls

* Adds module imports stub

* WIP state holder via *mut

* My take at the problem

* Brings back invoke and instantiate implementation details

* Removes redundant bound

* Code cleanup

* Fixes invoke closure

* Refactors FunctionExecutor to eliminate lifetime

* Wraps `FunctionExecutor::sandbox_store` in `RefCell`

* Renames `FunctionExecutor::heap` to `allocator`

* Wraps `FunctionExecutor::allocator` in `RefCell`

* Refactors FunctionExecutor to `Rc<Inner>` pattern

* Implements scoped TLS for FunctionExecutor

* Fixes wasmi instancing

* Fixes sandbox asserts

* Makes sandbox compile after wasmtime API change

* Uses Vurich/wasmtime for the Lightbeam backend

* Uses wasmtime instead of wasmi for sandbox API results

* Refactors sandbox to use one  of the execution backends at a time

* Fixes wasmtime module instantiation

* TEMP vurich branch stuff

* Adds wasmer impl stub

* Adds get global

* Fixes warnings

* Adds wasmer invoke impl

* Implements host function interface for wasmer

* Fixes wasmer instantiation result

* Adds workaround to remove debug_assert

* Fixes import object generation for wasmer

* Attempt to propagate wasmer::Store through sandbox::Store

* Wraps `sandbox::Store::memories` in `RefCell`

* Moves `sandbox::instantiate` to `sandbox::Store`

* Eliminate `RefCell<memories>`

* Implements `HostState::memory_get/set`, removes accidental `borrow_mut`

* Fixes sandbox memory handling for wasmi

* Fix memory allocation

* Resets Cargo.lock to match master

* Fixes compilation

* Refactors sandbox to use TLS for dispatch_thunk propagation to wasmer

* Pass dispatch thunk to the sandbox as a TLS

* Initialize dispatch thunk holder in `SandboxInstance`

* Comment out Wasmtime/Lightbeam sandbox backend

* Revert wasmtime back to mainstream

* Adds SandboxExecutionMethod enum for cli param

* Cleanup sandbox code

* Allow wasmi to access wasmer memory regions

* More cleanup

* Remove debug logging, replace asserts with runtime errors

* Revert "Adds SandboxExecutionMethod enum for cli param"

This reverts commit dcb2b1d3b54145ab51ad2e3fef0d980ba215b596.

* Fixes warnings

* Fixes indentation and line width

* Fix return types condition

* Puts everything related under the `wasmer-sandbox` feature flag

* Fixes warnings

* Address grumbles

* Split instantiate per backend

* More splits

* Refacmemory allocation

* Nitpicks

* Attempt to wrap wasmer memory in protoco enforcing type

* Revert renaming

* WIP wasm buffer proxy API

* Reimplement util::wasmer::MemoryRef to use buffers instead of memory slices

* Adds WasmiMemoryWrapper and MemoryTransfer trait

* Refactor naming

* Perform all memory transfers using MemoryTransfer

* Adds allocating `read`

* Adds comments

* Removes unused imports

* Removes now unused function

* Pulls Cargo.lock from origin/master

* Fix rustdoc

* Removes unused `TransferError`

* Update Cargo.lock

* Removes unused import

* cargo fmt

* Fix feature dependency graph

* Feature should flow from the top level crate
	* We should not assume a specific workspace structure
* sc-executor-wasmi does not use the feature
* sc-executor-wasmtime should not know about the feature

* Fix doc typo

* Enable wasmer-sandbox by default (for now)

It will be removed before merge. It is so that the benchbot
uses the wasmer sandbox.

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Revert "cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs"

This reverts commit d713590ba45387c4204b2ad97c8bd6f6ebabda4e.

* cargo fmt

* Add ci-check to prevent wasmer sandbox build breaking

* Run tests with wasmer-sandbox enabled

* Revert "Run tests with wasmer-sandbox enabled"

This reverts commit cff63156a162f9ffdab23e7cb94a30f44e320f8a.

Co-authored-by: Sergei Shulepov <[email protected]>
Co-authored-by: Andrew Jones <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>

* initial idea

* update benchmark test to frame v2

* fix some errors

* fixes for elec phrag

* fix tests

* update extrinsic time and docs

* fix import

* undo extra changes

* helper function

* wrong way

* Update frame/benchmarking/src/utils.rs

Co-authored-by: Bastian Köcher <[email protected]>

* doesnt need encode/decode

* fix benchmark return

Co-authored-by: Bastian Köcher <[email protected]>

* remove the uselsss weight return type from election provider API

* fix everything, should be ready for final benchmark

* simplify on_init a bit furhter

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* remove unwraps

* fmt

* Update lock file

* whitelist block weight

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix warning

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* upgrade simnet tests to v8

* undo change by mistake

* add suggested changes

* rm un-used script

* echo config of simnet tests

* use proper identation and rm : to fix ci-lint

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <[email protected]>

* fix description for tets

* rm build for PRs from this branch

* dummy trigger ci

* rm content for future PR

Co-authored-by: Denis Pisarev <[email protected]>

* Split native executor stuff from wasm executor stuff

* Remove `native_runtime_version` in places

* Fix warning

* Fix test warning

* Remove redundant NativeRuntimeInfo trait

* Add a warning for use_native

* Run cargo fmt

* Revert "Add a warning for use_native"

This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.

* Make choosing an executor (native/wasm) an explicit part of service construction

* Add Cargo.lock

* Rename Executor to ExecutorDispatch

* Update bin/node/executor/src/lib.rs

Co-authored-by: Squirrel <[email protected]>

* Fix tests

* Fix minor node-executor error

* Fix node cli command thing

Co-authored-by: Squirrel <[email protected]>

* Comment out browser stuff

* Remove browser stuff

* Remove more wasm transport code

* Remove ExtTransport and rework how telemetry initialises.

* Change (most) wasm-timer using code to use std::time

* Rename CI-job

* Aura does not compile for wasm

* Remove testing in the browser on CI

* Update README

* Leave `StreamSink` be

* fmt

* Fix prometheus after hyper 14 upgrade

* Fix stupid mistakes

* Use 127.0.0.1

* Update utils/prometheus/Cargo.toml

Co-authored-by: Niklas Adolfsson <[email protected]>

Co-authored-by: Niklas Adolfsson <[email protected]>

* Remove the `native_executor_instance!` macro

* Add comment to test runner ex

* Fix comments

* RUSTSEC-2021-0076 bump libsecp256k1

libsecp256k1 allows overflowing signatures
https://rustsec.org/advisories/RUSTSEC-2021-0076



Changes were made to conform to libsecp256k1 version differences.

Closes #9356

* parse_standard_slice() -> parse_overflowing_slice()

* Added v2 host function for ecdsa_verify

* Add feature tag over helpers

* Added ecdsa_verify v2 to test runner

* PR feedback

- Spaces -> tabs
- renamed two helper functions

* Fixed imports after rebasing

* Bump rest of libsecp256k1 (and libp2p)

libp2p also uses libsecp256k1 so it is required to be bumped too, along
with all the version difference changes.

* Add version2 for ecdsa pubkey recovery

* libp2p rebase master fixes

* Fix test panic when non Behaviour event is returned

* Update bin/node/browser-testing/Cargo.toml

* Update primitives/core/src/ecdsa.rs

* Update primitives/core/src/ecdsa.rs

* Update Cargo.lock

Co-authored-by: Bastian Köcher <[email protected]>

Add block hash to call data of TaggedTransactionQueue_validate_transaction in light api remote call (#9554)

Co-authored-by: Bernardo Cardoso <[email protected]>

Bumps [ip_network](https://github.com/JakubOnderka/ip_network) from 0.3.4 to 0.4.0.
- [Release notes](https://github.com/JakubOnderka/ip_network/releases)
- [Commits](https://github.com/JakubOnderka/ip_network/compare/v0.3.4...v0.4.0

)

---
updated-dependencies:
- dependency-name: ip_network
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Upgrade hyper to 0.14.10

* fmt

* Enable required features

Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.  
Updated code related to deprecated warning on tracing-subscriber `scope()` 
( See https://github.com/tokio-rs/tracing/issues/1429 )

Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.3 to 0.9.0.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.3...v0.9.0

)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Upgrade jsonrpc to 0.18.0

I think this says all :P

* 🤦

* Fmt etc

* Fix tests

* Fix tests again...

* Better impl

* Revert "Tell dependabot to ignore jsonrpc-* updates (#9518)"

This reverts commit 6e0cd558.

* Stableize `seal_debug_message`

* Update changelog

* Enable more tests

* Cargo fmt

* CI: publish docs to gh-pages

* CI: debug

* CI: add CI image

* CI: chmod for dox

* CI: typo

* CI: remove ownership debug

* CI: unfixme

* Revert "CI: debug"

This reverts commit f058b739fbe8c7ba6a9932e991ead93121d67309.

* CI: build-rust-doc can not fail now

* CI: remove publish-s3-doc

* CI: chore

* CI: less needs

* CI: pwd ruins the prettiness

* CI: return needs

* Partially applied

* Everything builds, need to implement compact encoding as well.

* Fix some tests, add a ui test as well.

* Fix everything and everything.

* small nits

* a bunch more rename

* more reorg

* more reorg

* last nit of self-review

* Seemingly fixed the build now

* Fix build

* make it work again

* Update primitives/npos-elections/solution-type/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update primitives/npos-elections/solution-type/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* nits

* factor out double type

* fix try-build

Co-authored-by: Guillaume Thiolliere <[email protected]>

* reformat everything again

* manual formatting

* last manual fix

* Fix build

* general cleanup of staking tests

* fix fishy test

* fix one more fishy test

* some review comments

* remove dependency that is no longer required

* remove unused deps

Actually this will cost us another allocation, but before this wasn't
really safe. Assuming that we only need `size_of` bytes for the encoding
of the tx could have ended with an invalid encoding.

* Split native executor stuff from wasm executor stuff

* Remove `native_runtime_version` in places

* Fix warning

* Fix test warning

* Remove redundant NativeRuntimeInfo trait

* Add a warning for use_native

* Run cargo fmt

* Revert "Add a warning for use_native"

This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.

* implement "auto" database backend in client/db, in progress, #9201

* move fn supports_ref_counting from DatabaseSource enum to Database trait to make it work correctly for all types of dbs

* update kvdb_rocksdb to 0.13 and use it's new config feature  to properly auto start existing database

* tests for auto database reopening

* introduce OpenDbError to cleanup opening database error handling and handle case when database is not enabled at the compile time

* cargo fmt strings again

* cargo fmt strings again

* rename DataSettingsSrc to fix test compilation

* fix the call to the new kvdb-rocksdb interdace in tests to fix compilation

* simplify OpenDbError and make it compile even when paritydb and rocksdb are disabled

* cargo fmt

* fix compilation without flag with-parity-db

* fix unused var compilation warning

* support different paths for rocksdb and paritydb in DatabaseSouce::Auto

* support "auto" database option in substrate cli

* enable Lz4 compression for some of the parity-db colums as per review suggestion

* applied review suggestions

* Fix staking `rebond` weight refund


Comment

* use safe arithmetic

* comment

* Clarify how `ApiId` is being generated

* Clarify more