* Restore "Remove service, migrate all to service-new (#1630)"

i.e.
Revert "Revert "Remove service, migrate all to service-new (#1630)" (#1731)"

This reverts commit c68aee35

.

This allows us to get the changeset from #1630 into a new branch
which can be merged sometime in the future after appropriate burnin
tests have completed.

* remove ',)' from codebase outside of macros

* restore bdfl-preferred formatting

* attempt to improve destructuring formatting

* rename polkadot-service-new -> polkadot-service

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* remove unused import

* Update runtime/rococo-v1/README.md

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

This reverts commit af14ea54.

* import rococo into chain-spec

* make a few stabs at moving forward

* wip: rococo readme

* remove /service crate

- Move the chain-spec files to node-service
- update sufficient cargo files that polkadot-service-new builds
- not everything else builds yet

* wip: chase down some build errors in polkadot-cli

There's a lot more to go, but some progress has happened.

* make more progress getting polkadot-cli to build

* don't ignore polkadot.json within the res directory

* don't recreate pathbufs

* Prepare Polkadot to be used by Cumulus

This begins to make Polkadot usable from Cumulus.

* Remove old test

* migrate new_chain_ops fix from /service

* partially remove node/test-service

* Reset some changes

* Revert "partially remove node/test-service"

This reverts commit 7b8f9ba5.

* WIP: replace v0 ParachainHost impl with v1 for test runtime

This is necessary because one of the current errors when building
the test service boils down to:

the trait bound `polkadot_test_runtime::RuntimeApiImpl<...>`:
  `polkadot_primitives::v1::ParachainHost<...>` is not satisfied

This is WIP because it appears to be causing some std leakage into
the wasm environment, or something; the compiler is currently
complaining about duplicate definitions of `panic_handler` and `oom`.
Presumably I have to identify all std types (Vec etc) and replace
them with sp_std equivalents.

* fix test runtime build

it wasn't std leakage, after all

* bump westend spec version

* use service-new as service within cli

* to revert: demo that forwarding the test runtime to the real impl blows up

* Revert "to revert: demo that forwarding the test runtime to the real impl blows up"

This reverts commit 68d2f385.

* Revert "Revert "to revert: demo that forwarding the test runtime to the real impl blows up""

This reverts commit 04cb1cbf.

Might have just forgotten to disable default features

* More reverts

* MOARE

* plug in the runtime as the generic instantiation

This feels closer to a solution, but it still has problems: in particular,
it's assumed that Runtime implements all appropriate Trait traits,
which this one apparently does not.

* implement necessary traits to get the test runtime compiling

This is almost certainly not correct in some way; it really
looks like I need to mess with the construct_runtime! macro
somehow, to inject the inclusion trait's event type as a Event
variant. Still, better lock down this changeset while it all
compiles.

* add inclusion::Event as variant into Event enum

* implement unimplemented bits in kusama

* implement unimplemented bits in polkadot runtime

* implement unimplemented bits in westend runtime

* migrate client upgrades from master

* update test service with new node changes

* package metadata--that wasn't intended to be removed

* add parachains v1 modules to each runtime

It's not clear what precisely this does, but it's probably the right
thing to do.

* enable cli to opt out of full node features

* adjust rococo chainspec per example

https://github.com/paritytech/polkadot/blob/26f1fa47f7836ab4bee5d4aad127ebce748320dd/service/src/chain_spec.rs#L362



* try to fix Cargo.lock

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Andronik Ordian <write@reusable.software>

* Prepare Polkadot to be used by Cumulus

This begins to make Polkadot usable from Cumulus.

* Remove old test

* Reset some changes

* More reverts

* MOARE

* Don't use `unimplemented!`

* clean out v0 consensus crates

* remove service dependencies on old consensus code

* fix cli

* kill adder-collator

* bump Cargo.lock

* cargo.toml updates

* session and system

* more

* more

* more

* more

* more

* fix

* compiles

* fix tests

* fix more tests

* fix mock

* fix deleted space

* Update validation/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock

* update rococo

* remove unused warning

* update add benchmarks

* rename weight file

* forgot a file

* Update chain_spec.rs

* Revert "remove unused warning"

This reverts commit 4227cd0d

.

* fix merge

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update scale codec to latest version to fix bug in future rustc version

Companion of: https://github.com/paritytech/substrate/pull/6746

* 'Update substrate'

Co-authored-by: parity-processbot <>

* network bridge skeleton

* move some primitives around and add debug impls

* protocol registration glue & abstract network interface

* add send_msgs to subsystemctx

* select logic

* transform different events into actions and handle

* implement remaining network bridge state machine

* start test skeleton

* make network methods asynchronous

* extract subsystem out to subsystem crate

* port over overseer to subsystem context trait

* fix minimal example

* fix overseer doc test

* update network-bridge crate

* write a subsystem test-helpers crate

* write a network test helper for network-bridge

* set up (broken) view test

* Revamp network to be more async-friendly and not require Sync

* fix spacing

* fix test compilation

* insert side-channel for actions

* Add some more message types to AllMessages

* introduce a test harness

* impl ProvideInherent for InclusionInherent

* reduce import churn; correct expect message

* move inclusion inherent identifier into primitives

It's not clear precisely why this is desired, but it's a pattern
I've seen in several places, so I'm going this to be on the
safe side. Worst case, we can revert this commit pretty easily.

* bump kusama spec_version to placate CI

* copy sc_basic_authorship::{ProposerFactory, Proposer}

We have from the problem description:

> This Proposer will require an OverseerHandle to make requests via.

That's next on the plate.

* use polkadot custom proposer instead of basic-authorship one

* add some tests

* ensure service compiles and passes tests

* fix typo

* fix service-new compilation

* Subsystem test helpers send messages synchronously

* remove smelly action inspector

* remove superfluous let binding

* fix warnings

* add license header

* empty commit; maybe github will notice the one with changes

* Update node/network/bridge/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* add sanity check to only include valid inherents

* stub: encapsulate block production mechanics instead of copying them

The goal is to end up with something like what's in
validation::block_production::*, which encapsulates
basic block production mechanics. This is a better idea than
just straight-up copying those mechanics.

* partial implementation of propose fn

Doesn't actually compile yet; need to bring in some other
commits to ensure ProvisionerMessage is a thing, and also
figure out how to get the block hash given the current
context.

* fix compilation

* clear a few more compile errors

* finish fn propose

* broken: add timeout to proposal

* add timeout to proposal

* guide: provisioner is responsible for selecting parachain candidates

* implement ProvisionerMessage::RequestInherentData & update fn propose

* impl CreateProposer::init; clean up

* impl std::error::Error for Error

* document error-handling rationale

* cause polkadot-service-new to compile correctly

* Move potentially-blocking call from fn init -> fn propose

This means that we can wrap the delayed call into the same
timeout check used elsewhere.

* document struct Proposer

* extract provisioner data fetch

This satisfies two requirements:

- only applies the timeout to actually fetching the provisioner data,
  not to constructing the block after
- simplifies the problem of injecting default data if we could not
  get the real provisioner data in time.

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>

* Switch branch

* Return chain ops parts in new_chain_ops

* Remove where param from new_chain_ops

* Add task manager to new_chain_ops return

* Revert branch switch

* Revert "Revert branch switch"

This reverts commit 7c7900c0.

* network/test/src/lib: Adjust network worker polling

Companion for https://github.com/paritytech/substrate/pull/6552

.

* Fix adder parachain

* Fix collator tests

* Revert branch switch

* Bump everything

- Remove old migration code
- Reduce deposit requried for westend

* Reapply fixes

* Bump locl

* Fix for #6550

* Fix message

Co-authored-by: Ashley Ruglys <ashley.ruglys@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>

* network bridge skeleton

* move some primitives around and add debug impls

* protocol registration glue & abstract network interface

* add send_msgs to subsystemctx

* select logic

* transform different events into actions and handle

* implement remaining network bridge state machine

* start test skeleton

* make network methods asynchronous

* extract subsystem out to subsystem crate

* port over overseer to subsystem context trait

* fix minimal example

* fix overseer doc test

* update network-bridge crate

* write a subsystem test-helpers crate

* write a network test helper for network-bridge

* set up (broken) view test

* Revamp network to be more async-friendly and not require Sync

* fix spacing

* fix test compilation

* insert side-channel for actions

* Add some more message types to AllMessages

* introduce a test harness

* add some tests

* ensure service compiles and passes tests

* fix typo

* fix service-new compilation

* Subsystem test helpers send messages synchronously

* remove smelly action inspector

* remove superfluous let binding

* fix warnings

* Update node/network/bridge/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* fix compilation

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* New service initial commit

* More separation of the new and old services

* Fix review comments

* Adds polkadot.json

* Fix browser build

* Remove unused import

* Update node/service/src/lib.rs

Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>

* Remove duplicate json files

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Bump Substrate again.

* update kusama runtmie for ensure origin

* update polkadot runtime for ensure origin

* fix imports

* root only available for runtime benchmarks

* Fix lock file

* Bump Substrate

* Update lock

* Remove questionable feature-gating.

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Bump versions.

* Update lock

* Fix for new Substrate

* Fix

* Remove Utility constraint in NonTransfer

* Bump

* Fixes

* Allow limited utilities in proxy filters.

* Fix

* Add SudoBalances proxy restrictions

* Introduce generalised proxies to polkadot

* Introduce proxy to westend

* Add proxy to Kusama.

* Fix

* Identity

* Bump spec

* Update runtime/polkadot/src/lib.rs

Co-authored-by: kaichao <kaichaosuna@gmail.com>

* Fix runtime

* Bump runtimes

Co-authored-by: kaichao <kaichaosuna@gmail.com>

* Bump

* Bump

* Bump

* Test CC1 spec

* Adaptive default chainspec

* Strip -pre

* Update to next CC1 iteration

* Raw chainspec

* Update claim statements

* Update URLs

* Minor tweak to make enum agree with URL

* final html multihashes

* New hashes

* Version

* updated SAFT statement hash

* Update runtimes; this relies on substrate #6131

* Bump

* Share transaction filterer and fix Kusama

* Warning

* Gah!

* Tidy

* Westend fixes

* Westend fix

* Tweak constants

* Implement TCF stuff

* Fix

* Warning

* 15 minute lookahead

* 15 minute lookahead in Polkadot

* update max transactions

* Enable utility, disable vested_transfer

* Update runtime/common/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Filter calls for validate unsigned also

* Deduplicate

* Fix

* Introduce Polkadot (mainnet) chainspec

* Fix naming.

* Enable indices and fix comment.

* Fix compilation

* Enable indices and fix comment.

* polkadot: babe: enable secondary VRF slots

* Test JSON

* Allow set_heads

* Fix

* Raw chain spec added

Co-authored-by: keorn <github@i.keorn.org>
Co-authored-by: NikVolf <nikvolf@gmail.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
Co-authored-by: André Silva <andre.beat@gmail.com>

* Introduce mandatory statement signing into claims

* Introduce SignedExtension

* Tests passing

* Bump runtime version

* Bump version, fix test

* Test for validate

* Another couple of tests

* Enable PrevalidateAttests on Polkadot

* Enable PrevalidateAttests on Polkadot

* Fix build

* Fixes

* More fixes

* Fix bench tests

* Fix & test Preclaim clobbering.

* Fix for errant claim logic

* Add test

* Update tests, always use Vec as input

* mint_claim can add signature, some_benchmarks

* Add claim with statement test

* finish benchmarks

* put the correct number of claims with benchmarks

* fix compiler warning

* Update weights

* Weight comments for validation

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Bump Westend runtime version

* Bump Substrate

* Prep for release of 0.7.32

* Update lock

* WIP

* collator

* fix missing reference

* Make collator builder more generic

* all clear

* revert start colator fn

* fix return type of start_collator

* sc-consensus

* bump back to master

* Update collator/src/lib.rs

Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>

* Update collator/src/lib.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* uncomment type MaxIterations

* reactivate append storage feature

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* add storage_append

* Bump versions

Co-authored-by: Gav Wood <gavin@parity.io>

* Switch from using the substrate_test_runtime Extrinsic to the polkadot_test_runtime one

* Copy genesismap into test-runtime

* Add UncheckedExtrinsics

* Fix tests :^)

* Remove unused functions from genesismap

* DRY, clean up

* Clean up

* Update service/src/grandpa_support.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix indentation

* Update runtime/test-runtime/src/genesismap.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Release 0.7.30

* Update Substrate & libp2p

* Updated for the new db

* Bumped version

* Sort out features

* Update to substrate master

* Release 0.7.29

* Update `Cargo.lock`

* Initial draft

* More work

* Build

* Docs

* Insert westend keys

* Add badBlock to fork from old chain

* Updated spec to reset westend

* Use raw spec

* Fix spec format and use westend2 for both id's

* Correct public key for bootnode 3

* Build

* Extra space

* Fix build

* Lock

* Update lock

* Fixes

* Fix for he startup text

* Bump

Co-authored-by: Gav Wood <gavin@parity.io>

* Fixes for democracy using Scheduler

* Revert branch change

* Updates for the new democracy

* Introduce progressive democracy

Also move to "master" branch of Substrate ready for continuous
sync with Substrate master.

* bump version to v0.7.27

* update to latest substrate polkadot-master

* bump runtime impl_version

* grandpa: add voting rule to pause new votes for a period

* grandpa: increase delay

* grandpa: parse custom pause delay from cli

* grandpa: log scheduled pause on startup

* grandpa: rename parameter to grandpa_pause

* grandpa: make pause voting rule generic on block

* grandpa: add test for pause voting rule

* grandpa: add hardcoded pause

* collator: fix test compilation