* Implement ext_ hashes for contracts (issue #5258)

* load cryto hash .wat from raw string literal instead of file

* update .wat contents for testing crypto hashes

* remove unnecessary 'static

* fix bug in input (call_indirect required 1+ at least it seems)

* no longer use scratch buffer for crypto hash functions

* improve doc comments of ext_ hash functions

* remove unnecessary comment in .wat test file

* add return value (const 0) to contract test to hopefully enable result buffer

* fix bug in contract assertion

* implement proper output_len in contract

* implement proper test for crypto hashes

* bump spec_version 238 -> 239

* fix COMPLEXITY description

* remove final invalid instances of scratch buffer from docs

* Make sure frame examples compile for wasm

This makes sure that `frame-example` and `frame-example-offchain-worker`
compile for wasm.

This also fixes compilation for these crates. The offchain worker
example doesn't use serde-json anymore as that is too heavy and breaks
`no_std` compilation.

* Apply suggestions from code review

Co-Authored-By: Nikolay Volf <[email protected]>

Co-authored-by: Nikolay Volf <[email protected]>

* Remove the #[doc(hidden)] reexports of sc_network

* Fix tests not building

* Start benchmarks

* try_origin or root

* More benches

* stuck

* Custom trait functions for benchmarks

* finish benchmarks

* Bump impl

* More comments

* Bump spec

* Remove import

* Update frame/elections-phragmen/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* Update frame/support/src/traits.rs

Co-Authored-By: thiolliere <[email protected]>

* Fix merge

Co-authored-by: thiolliere <[email protected]>

* Factor out can_set_code

The motivation for this feature is parachain runtime upgrades, which
happen in two different moments: the initial call, when the checks
should be performed, and at the actual upgrade block. It's much
better and more maintainable to just call `can_set_code` in the
function than to manually copy those checks.

I'm not entirely thrilled with the interface, which requires
cloning the entire code vector in `set_code`. However, it looks
like something in `decl_module!` does not play nicely with
references in its function arguments. If `code` has a lifetime,
it _must_ be named, which gives us this signature.

```rust
pub fn can_set_code<'a>(origin, code: &'a [u8]) {
```

Unfortunately, attempting to compile with that signature generates
a very large and baffling collection of build errors, so I decided
to abandon that path for now.

* make can_set_code non-dispatchable per PR revew

Not only can we now borrow the `code` slice, but we also no longer
need to worry about people sending a `can_set_code` transaction because
of some misunderstanding.

* move into existing impl block

* Initial benchmarking setup

* Add keystore

* validate unsigned

* Update frame/im-online/src/benchmarking.rs

Co-Authored-By: Marcio Diaz <[email protected]>

* Fix verify_unsigned benchmark

* add variable for teting the external addresss length

* Update frame/im-online/src/benchmarking.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Update utils/frame/benchmarking-cli/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Marcio Diaz <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Benjamin Kampmann <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Remove migration code.

* Fix cargo

* Bump spec version

* Prompt error if vanity pattern is not valid base58

* Update bin/utils/subkey/src/vanity.rs

* Add tests for validating pattern of subkey vanity

Co-authored-by: Bastian Köcher <[email protected]>

* switching to released wasmtime

* update depdencies in general

* Update fdlimit

* Update parity common deps

* Also update schnorrkel

* update kvdb-rocksdb

* update further dependents

* also update primitive types

* update cargo.lock

* update merlin

* Bumping evm version

* Don't queue duplicate blocks

* Keep queue_blocks on restart

env_logger::init() can lead to panics on concurrent test execution.

See: https://gitlab.parity.io/parity/substrate/-/jobs/422283

* grandpa: support for hard forking any pending standard changes

* grandpa: expose authority_set_hard_forks in block import constructor

* grandpa: don't break the public api

* client/network-gossip/src/bridge: Finish when network even stream closes

Previously within `<GossipEngine as Future>::poll` one would poll the
`network_event_stream` ignoring all messages other than
`Poll::Ready(Some())`. Ignoring `Poll::Ready(None)` leads to a panic on
the next poll of the stream, gien that it is not fused.

By design `network_event_stream` does not close unless an unbounded send
into it fails, or the `NetworkWorker` gets shut down.

> The stream never ends (unless the `NetworkWorker` gets shut down).
> (client/network/src/service.rs)

An `unbounded_send` to fail on an unbounded channel is unlikely. The
`NetworkWorker` shutting down is not unlikely. In such case the
`GossipEngine` should shut down as well.

With this patch a `<GossipEngine as Future>` finishes on
`Poll::Ready(None)` returned from `network_event_stream`.

* client/finality-grandpa/communication: Error on gossip engine finished

Have `<NetworkBridge as Future>::poll` return `Poll::Ready(Err)` instead
of `Poll::Ready(Ok)` to be consistent with the handling of the neighbor
packet worker stream and the gossip validator report stream. Both `Err`
as well as `Ok` shut down the `NetworkBridge` as well as the
`VoterWorker`.

* client/network-gossip/src/bridge: Add regression test

* client/network-gossip: Move substrate test client to dev dependencies

* client/network-gossip: Remove TODO

Addressed in a follow up pull request.

* client/network-gossip/bridge: Put match on newline after loop

* client/finality-grandpa/src/observer: Fix regression test

Make sure the event stream sender side is not dropped till the end.

* Adding missing metadata for benchmark-pallet

* Missed to bump a few

* release on roling tag

* Wasm executors were missed

* Version bump

* Split generate_changelog.sh into separate script

Can be run in the format `generate_changelog.sh $previous_version $version`.

* remove early exit from publish_draft_release.sh

* adding changelog

* ci: change last_github_release to also find pre-releases

Co-authored-by: Martin Pugh <[email protected]>

If polling encounters a `Poll::Pending` we need to return it instead of
continuing the loop which may turn it into a blocking operation, causing
problems with executors.

This reverts commit 1b8b62b1.

* Print non-output messages to stderr in subkey

* Update bin/utils/subkey/src/main.rs

Co-authored-by: Bastian Köcher <[email protected]>

if statement logic for recently sent block was inverted.

* make sure return ready iterator once state is updated

* update sc_basic_authorship tests

* update node tests

* fix manual seal

* actually fix service test

* add tests

* Update client/basic-authorship/src/basic_authorship.rs

Co-Authored-By: Tomasz Drwięga <[email protected]>

* helper function

* review suggestions

* warning and continue

* add debug log

* use futures::chennel::oneshot

* use declaration bound

* no option for updated_at

* no allocation

* ready_at / ready

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Fix vesting bench starting block.

* Set subs s+1 to hit worst path.

* Remove unused origin.

* Apply review suggestions.

* starting bench

* More

* more

* Payout Validator

* Give each validator exactly n nominators

* Update with test

* Try to add accounts to chain spec

* Undo changes to chainspec

* Payout nominator

* Rebond and Reap Stash

* Set history depth

* fix smelly code

* cancel deferred slash

* new_era bench

* do_slash benchmark

* Add features

* undo extrinsic move

* lower

* Update new era

* Update benchmarking.rs

* whitespace

* Apply suggestions from code review

Co-Authored-By: Kian Paimani <[email protected]>

* fixes

* nit

* Refactor tests, initial code

* Move session benchmarks to avoid cyclic deps

* Update lib.rs

* Fix warnings

* Move impl

* Update to do random nominator allocation

* add feature to benchmark pallet

* Remove extra stuff

* Update based on feedback

* Less intrusive

* Remove `transfer_idle_users`

* remove again

* unused dep

* test feature flag

* Update to latest substrate

Co-authored-by: Kian Paimani <[email protected]>

With this patch forward this will be the only way for
a contract to destroy itself. This patch therefore changes
the semantics of all other contract initiated balance
transfers to fail if they would bring the caller below the
existential deposit.

* client/network/src/protocol: Start Prometheus metric help with capital

* client/network/src/protocol: Differentiate metric status as label

Prometheus query language is powerful through its multi-dimensional data
model. Metric names are hirarchical whereas labels enable data to become
multi-dimensional.

Exposing the justification of finality-proof status as a label allows
for more powerful queries.

* client/network/src/protocol: Remove 'Total' from non counter metric help

The word 'total' is reserved for accumulating counters. Counters have to
be monotonically increasing. `obsolete_requests` can decrease, thereby
it is defined as a `Gauge` and not a `Counter`.

For more details on metric naming see
https://prometheus.io/docs/practices/naming/

* System.BlockHash

* Fix hash

* Introduce K/V iteration in all _concat maps

Also move across:
- System.Account (blake2_128_concat)
- Balances.Locks (twox_64_concat)
- ElectionsPhragmen.VotesOf (twox_64_concat)
- ElectionsPhragmen.StakeOf (twox_64_concat)
- Identity.IdentityOf (twox_64_concat)
- Identity.SubsOf (twox_64_concat)
- Society.Payouts (twox_64_concat)
- Session.NextKeys (twox_64_concat)
- Identity.SuperOf (blake2_128_concat)
- Session.KeyOwner (blake2_128_concat)
- Society.SuspendedCandidates (twox_64_concat)
- Society.SuspendedMembers (twox_64_concat)
- Society.Vouching (twox_64_concat)
- Society.Strikes (twox_64_concat)
- System.EventTopics
- Balances.Account

* Build fixes

* Ensure migration happens in correct order

* Staking.*

* Vesting.* Offences.*

* Democracy.*

* Babe.* Collective.*

* Grandpa.*

* Assets.* Benchmark.* Contracts.* Elections.* Asset.* Nicks.*

Also introduce real account list

* ImOnline.*

* Treasury.*

* Recovery.*

* Final bits.

* Docs

* Fix one test

* Fix test

* All passing except the UI tests

* Remove linked_map part 1

* Remove linked_map

* Some iterator utils for double maps.

* Remove old migrations

* Introduce tombstone for LinkedMap type

* Migration for genesis hash

* Fix build

* Fix hash

* Rename Map is_linked -> unused, keeping backwards compat (#5256)

* Update frame/balances/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update frame/elections/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Remove old migration code.

* Update frame/system/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update bin/node/runtime/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Fix hash

* fix session migration

* Fix watning

Co-authored-by: Jaco Greeff <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>

* fork-tree: prune returns all pruned node data

* epoch-changes: split EpochHeader vs epoch data

* EpochChanges::viable_epoch and add missing comments

* Incoperate the new epoch_changes interface for BABE

* Fix BABE tests

* Fix fork-tree pruning issue

* Fix tests

* Fix pruning algorithm

* fork-tree: implement map function for mapping one value type to another

* Add migration script for new epoch changes scheme

* Update utils/fork-tree/src/lib.rs

Co-Authored-By: André Silva <[email protected]>

* Update client/consensus/slots/src/lib.rs

Co-Authored-By: André Silva <[email protected]>

* Remove authorities_len.is_none check, which is duplicate of unwrap_or(false)

* Update client/consensus/epochs/src/lib.rs

Co-Authored-By: André Silva <[email protected]>

* Update client/consensus/epochs/src/lib.rs

Co-Authored-By: André Silva <[email protected]>

* No trailing ; for return statement

* Use VERSION_KEY for migration

* Fix issues that removed nodes are not properly added into removed list

* Add comments indicating end_slot is non-inclusive

* fork-tree: use &mut F for map type declaration

* Add tests for v0 epoch_changes migration

* Fix babe RPC tests

Co-authored-by: André Silva <[email protected]>