* Upgrade Polkadot & Substrate

* Update test/parachain/src/command.rs

Co-authored-by: Cecile Tonglet <[email protected]>

Co-authored-by: Cecile Tonglet <[email protected]>

* Start

* Fix compilation

* Fix chainspec

* Don't set best 2 times for the same block

* Check the status of a block before building on it

* Check that the block exists before setting it as the new best

* Reorder code

* Fork choice depends on sync status

* Switch branch again

* Switch to latest Substrate & Polkadot

* Update again

* Start by replacing branch names and set `DownwardMessage`

* Add the upward-message crate

* Add Kusama & Polkadot

* More work on getting the upward messages working

* Fix build

* Begin to integrate it into the test Parachain

* Update

* Make everything compile again

* Switch to westend and print parachain account on startup

* Use MultiSignature etc

* Fix validate block

* Some downward messages work

* Update git reference

* More downward messages integration

* Update test runtime for downward messages

* Enable downward message handler and withdraw send tokens

* Add some docs

* Begin to implement simple XCMP

* More work

* Fixes and make parachain id configurable

* Make parachain ID be part of the genesis

* Finishing the XCMP message demo

* Update and fixes tests

* Update branch

* Update to latest Substrate & Polkadot

* Replace --unsafe-rpc-export with --unsafe-rpc-external

* Add --rpc-methods=Unsafe

Documented in substrate 24486f52929e9e518eeccbc6ad6da70e9e5bdf8a

* typos

* more typo

* fixed rpc expose

* Disable the integration test

Co-authored-by: Cecile Tonglet <[email protected]>

Fixes #57

* Initial commit

Forked at: 6c74beab
Parent branch: origin/master

* Reject blocks without justification

* Revert "Reject blocks without justification"

This reverts commit ee60e12097939d4ccfe987a71db9a876319ae5ba.

* WIP

Forked at: 6c74beab
Parent branch: origin/master

* WIP

Forked at: 6c74beab
Parent branch: origin/master

* WIP

Forked at: 6c74beab
Parent branch: origin/master

* CLEANUP

Forked at: 6c74beab
Parent branch: origin/master

* WIP

Forked at: 6c74beab
Parent branch: origin/master

* WIP

Forked at: 6c74beab
Parent branch: origin/master

* CLEANUP

Forked at: 6c74beab
Parent branch: origin/master

* WIP

Forked at: 6c74beab
Parent branch: origin/master

* WIP

Forked at: 6c74beab


Parent branch: origin/master

* Move HeadData to primitives

* Update network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* CLEANUP

Forked at: 6c74beab
Parent branch: origin/master

* fix

* CLEANUP

Forked at: 6c74beab


Parent branch: origin/master

* messages

* for the greater good

* Update primitives/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* Initial commit

Forked at: ab51fe9a
Parent branch: origin/master

* Consistency fix for repositories in Cargo.toml

* Inverse

This ensures that the crate compiles for `no_std`. Besides this, there
are some fixes to the crate code itself.

* fix test collator compilation failure

Closes #88.

* copy method instead of using feature

Using a feature gate like "test-features" is problematic because
it is leaky: depending on situational considitons such as the
current working directory when compilation is attempted, the
feature may or may not be applied, which makes success inconsistent.
It's simpler in this case to copy a dozen lines of code than to
work out all the issues with test features.

* docker script shoudld work on cumulus-branch (default)

* Add storage_append to runtime

Co-authored-by: Cecile Tonglet <[email protected]>

* Initial commit

Forked at: 66b5c97c


Parent branch: origin/master

* Add storage_append to runtime

* Use sp_std::vec!

* Reduce # of allocations

* Update runtime/src/validate_block/implementation.rs

Co-authored-by: Bastian Köcher <[email protected]>

* fix test collator compilation failure

Closes #88.

* copy method instead of using feature

Using a feature gate like "test-features" is problematic because
it is leaky: depending on situational considitons such as the
current working directory when compilation is attempted, the
feature may or may not be applied, which makes success inconsistent.
It's simpler in this case to copy a dozen lines of code than to
work out all the issues with test features.

* Align trie iteration to substrate.

* Can use expect on trie error, remove debug_assert

* Initial commit

Forked at: d4562c3a
Parent branch: origin/master

* Change substrate and polkadot branch to cecton-cumulus-branch

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* update .editorconfig

* should probably be --dev

* formatting

* Change substrate & polkadot branch to cecton-keep-unpinned-para-blocks

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* Update polkadot & substrate branches

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* Some fixes

* Updated polkadot

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* WIP

Forked at: d4562c3a
Parent branch: origin/master

* Cargo.lock

* WIP

Forked at: e81bbaae
Parent branch: origin/master

* WIP

Forked at: e81bbaae
Parent branch: origin/master

* Update substrate & polkadot branches

* Clean-up

* WIP

Forked at: e81bbaae
Parent branch: origin/master

* WIP

Forked at: e81bbaae
Parent branch: origin/master

* WIP

Forked at: e81bbaae
Parent branch: origin/master

* WIP

Forked at: e81bbaae
Parent branch: origin/master

* WIP

Forked at: e81bbaae


Parent branch: origin/master

* Use JustifiedBlockAnnounceValidator

* debug

* Revert "debug"

This reverts commit 2404121b840984d6ffc1012ccec1e6a784fa236c.

* Revert "Use JustifiedBlockAnnounceValidator"

This reverts commit 33e2efef21a1553c294b2bc23fca8b08bf77be2b.

* Revert branch to cumulus-branch

* clean-up

* clean-up

* cleanup

* cleanup

* Update branches

* debug

* Cancel previous task when new one is created

* Remove stream-cancel

* Clean-up

* cleanup

* cleanup

* Fix merge issue

* Update polkadot_chainspec.json

* Update network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Use closure instead of NetworkService to announce_block

* doc

* WIP

Forked at: e81bbaae


Parent branch: origin/master

* Move disable announcement to its own function with doc

* Change substrate and polkadot branch to cumulus-master

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Initial commit

* Update to 3e65111

* Add cfg_attr ... no_std

* Fix version

* WIP: add really simple validate_block insert validity check

* WIP: create a parachain upgrade pallet

This pallet will eventually make life much easier for people attempting
to upgrade parachains on their validator nodes, but for the moment,
key sections remain unimplemented while dependency details are worked
out.

* Implement basic admin-auth pallet functionality.

This compiles, which means it's probably mostly correct. However,
it's pretty far from being finished. Work yet to come:

- Integrate with the democracy pallet somehow to eliminate the
  requirement for the root user to initiate this process.
- Figure out what to do in the event that the parachain blocks
  and relay chain blocks get out of sync / delayed.
- Add testing... somehow. (What's reasonable to test?)

Open questions:

- Is the block number parameter in `on_initialize` the parachain
  block number, or the relay chain block number? If, as I suspect,
  it's the parachain block number, how do we deal with the fact that
  the real upgrade should happen on a very specific parachain block
  number?
- In line 68, is it reasonable to use `if n >= apply_block`, or should
  that line require strict equality?
- Is it reasonable to store/retrieve `CurrentBlockNumber` on every block,
  or is there a more streamlined way to pass that data between functions?
  Maybe it can be inserted into `struct Module` somehow?
- Can we somehow parametrize ValidationUpgradeDelayBlocks by T in
  order to eliminate the `.into()` call?

* use a better storage name

* Add checks ensuring runtime versions increase

Largely cribbed from https://github.com/paritytech/substrate/blob/a439a7aa5a9a3df2a42d9b25ea04288d3a0866e8/frame/system/src/lib.rs#L467-L494

* fix tests

* WIP: add tests from frame/system set_code

Currently doesn't build: line 230 is the problem. Removing or
commenting that line results in the new tests failing due to a
missing block number. Adding it, in an attempt to fix the problem,
fails to compile with this error:

```
   Compiling parachain-upgrade-pallet v2.0.0 (/home/coriolinus/Documents/Projects/coriolinus/parachain-upgrade-pallet)
error[E0599]: no function or associated item named `set_block_number` found for struct `Module<tests::Test>` in the current scope
   --> src/lib.rs:230:21
    |
47  | / decl_module! {
48  | |     pub struct Module<T: Trait> for enum Call where origin: T::Origin {
49  | |         // Initializing events
50  | |         // this is needed only if you are using events in your pallet
...   |
100 | |     }
101 | | }
    | |_- function or associated item `set_block_number` not found for this
...
230 |               System::set_block_number(123);
    |                       ^^^^^^^^^^^^^^^^
    |                       |
    |                       function or associated item not found in `Module<tests::Test>`
    |                       help: there is a method with a similar name: `current_block_number`
    |
    = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: aborting due to previous error

For more information about this error, try `rustc --explain E0599`.
error: could not compile `parachain-upgrade-pallet`.
```

That error is very weird, because the function does in fact exist:
https://github.com/paritytech/substrate/blob/a439a7aa5a9a3df2a42d9b25ea04288d3a0866e8/frame/system/src/lib.rs#L897



* cause tests to pass

Turns out that in fact there was some setup required in order to
get everything testing properly, but now we have a set of passing
unit tests which test some of the more common error cases.

* Add overlapping upgrades test

This currently fails, and I don't yet know why. TODO!

* Fix some logic errors

- In particular, only remove the pending validation function from
  storage when it's time to apply it.
- Don't store our own copy of the current block number.

* WIP: delegate most code upgrade permissions checks

They're defined in System::can_set_code, so may as well use them.

Unfortunately, the tests all fail for this right now, and I don't
yet understand why. Pushing to get immutable line number references.

* fix tests after delegating runtime checks to can_set_code

* WIP: events test

Right now, the events struct doesn't seem to contain enough information
to validate the particular events that we should have fired. Almost
certainly, this is a usage error on my part.

* fully initialize and finalize in event test

This doesn't change the results, though.

* fix events test

This was complicated to figure out. For the record, testing events
requires:

- a private module which publicly exports the crate's event type
- impl_outer_event macro to construct an enum wrapping the event
  types from the listed modules
- system::Trait and local Trait both declare `type Event = TestEvent;`
- (optional) group events with `System::<Test>::initialize(...)` and
  `System::<Test>::finalize()` calls.

It's not entirely clear why both events show up during the initialization
phase; my suspicion is that this is an artifact of not mocking a
particular extrinsic, such that they end up in initialization by default.

* cleanup and move crate to subdirectory

this prepares us to merge this pallet into the cumulus repo

* provisionally incorporate polkadot changes to hook everything together

This feels like the logical next step, and compiles, at least. Still,
there are some big TODOs remaining:

- merge the polkadot PR upstream and reset the polkadot branch in
  `runtime/Cargo.toml`
- in `runtime/src/validate_block/implementation.rs:116`, we should
  almost certainly return `Some(something)` sometime. When, precisely,
  and how to keep track of the appropriate data are all still open
  questions.

* WIP: further updates to work with the polkadot implementation

Hopefully we can upstream `ValidationFunctionParams` into the
polkadot trait defs so we can just copy the struct out of
`ValidationParams`, but no huge loss if not.

This should be more or less everything required at this level.
Next up: fix up `pallet-parachain-upgrade` so it reads from
`VALIDATION_FUNCTION_PARAMS` to determine upgrade legality and
upgrade block, and writes to `NEW_VALIDATION_CODE` when appropriate.

* update pallet-parachain-upgrade appropriately to handle new expectations

Implements the pallet side of the new flow. Basic tests appear to work.

Next up:

- make the "real blob" test work
- add a bunch of additional tests of all the corners

* remove test which set a real WASM blob

This test didn't directly test any of the code in this pallet;
it existed because we were just copying tests out of the substrate
implementation. Now that we have real code of our own to test,
(and because it's not compatible with the `BlockTests` abstraction,)
we don't need that test anymore.

Also added a `Drop` impl to `BlockTests` ensuring they get run at
least once.

* add test that storage gets manipulated as expected

* get validate_block tests compiling again

* Check validation function size against polkadot parameters

Generate a user-handlable error message if it's too big, so that
nothing panics later.

* demonstrate that block tests run

* don't actually store any magic values in parachain storage

We're allowed to use it as a transport layer between validate_block
and the parachain upgrade pallet, but actually editing it or, in
particular, attempting to persist data there which didn't originate
in the extrinsic, breaks things.

This means that we can't keep the :code insertion check, because
the validate_block layer doesn't know when it is legal to actually
upgrade the parachain. However, the rest of the features survive,
and all tests currently pass, so I'm counting it as a win.

Next up: look into adding an inherent which publishes the
ValidationFunctionParams struct to arbitrary pallets.

* Add reference to polkadot_client to Collator

This enables us to get the validation function parameters at
runtime, which unblocks creating an inherent containing them.

* remove unused imports

* Remove VFPX; build VFP from existing data structures

I almost don't want to know how long both global_validation
and local_validation have existed in the produce_candidate
function signature; they were precisely what I needed,
without needing to add anything to the Collator struct at all.

Oh well, at least I noticed it before putting the PR up for review.

Next up: create a proper inherent definition for the
ValidationFunctionParams.

* WIP: add cumulus-validation-function-params crate

Modeled on the substrate timestamp crate.
It's not currently obvious to me why it is desirable to publish
an entire crate for what amounts to a single const definition;
going to ask about that.

* refactor: get rid of validation-function-params crate

Everything about the VFPs has been moved into a module of runtime

* WIP: get VFP from inherent, when possible

Doesn't compile for weird trait errors; probable next steps: just
copy over the relevant code directly.

* ensure VFPs are available during block production and validation

* cleanup in preparation for review request

* Copy cumulus-primitives crate from bkchr-message-broker

That branch is visible as #80; this message copies the crate as of
d4b2045573796955de4e5bf8f74b6c48b44c3bee.

This isn't even a cherry-pick, because the commit which introduced
the primitives crate also did some work which from the perspective
of this PR is irrelevant. With any luck, by coping the crate directly,
there won't be too many merge conflicts when the second of these
open PRs is merged.

* move mod validation_function_params to cumulus_primitives

There is some very weird behavior going on with cargo check: every
individual crate checks out fine, as verified with this invocation:

for crate in $(fd Cargo.toml | xargs dirname); do
    if [ "$crate" == . ] || [[ "$crate" == *test* ]]; then continue; fi;
    name=$(toml2json "$crate/Cargo.toml" | jq -r '.package.name')
    if ! cargo check -p "$name" >/dev/null 2>/dev/null; then
        echo "failed to build $name"
    fi
done

However, `cargo check .` no longer works; it is suddenly demanding
clang in order to build an indirect dependency. I'm not going to
keep messing around with this anymore; it's more profitable for the
moment to knock out the rest of the requested changes. Still, this
behavior is weird enough that I really don't have any idea why
it might be happening.

* convert indentation to tabs

* rename parachain upgrade pallet per PR review

* use compact form for dependencies

* remove pallet readme

Move pertinent documentation into pallet's rustdoc.

* Add weight data in compliance with updated substrate requirements

The substrate API changed, so now we _have_ to invent some kind of
weight data for the dispatchables. This commit does so, with the
caveat that the numbers are pulled straight out of nowhere. Benchmarking
remains a TODO item.

* use anonymous fatal error type for brevity

* Create, use a Call for setting VFPs

Modeled on Timestamp; makes the ProvideInherent impl work much better.

* fix pallet tests

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

* fix formatting

* add license header

* refactor primitive inherents / keys into appropriate modules

* impl From<(GlobalValidationSchedule, LocalValidationData)> for ValidationFunctionParams

* extract inherent data vfp injection into a function

* collapse parachain dependency into compact form

* always store vfps under same storage key

* fix docs

* use minimum weight for VFP inherent

* rename module methods for clarity

* fix tests: set_code -> schedule_upgrade

* Apply pending validation function at inherent creation, not init

Initialization happens before inherent creation, which means that
at the time of `on_initialize`, the VFPs for the current block
have not yet been loaded. This is a problem, because it means that
updates would happen one block late, every time.

Moving that logic into inherent creation means that we always have
current information to work with.

* typo: default_features -> default-features

* do not panic in create_inherent

* revert f741cf0f2bc; don't change behavior, but use correct spelling

* move block initialization logic from inherent creation into the inherent

* re-disable default features

It is very difficult to come up with a coherent theory of what's
going on with these default features. Builds were all broken as of
3eb1618. Renaming them in f741cf0 seemed to fix that behavior.
Then they broke again locally, prompting aaee1c0. This commit
restores the status quo as of f741cf0; with any luck, the build
will succeed both locally and in CI.

* regenerate Cargo.lock

This updates several packages, but by inspection, they are all published
crates from crates.io; by semver, this should not cause any behavioral
changes.

This also updates the lockfile format to the new format.

The point of this commit is to deal with the fact that `sc-client`
no longer exists.

* fix checks given new dependencies

Appropriate weight declarations have changed; this follows them,
still using timestamp examples.

Note that these weights are almost certainly wrong.

* fix tests given new dependencies

* add another check preventing block VFPs from contaminating validity checks

* Add OnValidationFunctionParams trait so other modules can callback

There isn't yet an obvious use case for other modules to get the
validation function params from this one, but we may as well support
it on the off chance.

* Add get_validation_function_params

This getter allows other modules to simply get the validation
function parameters, if they have been updated for this block.
Otherwise, it returns None, and they can try again later.

* upgrade substrate: panic on div by 0

* Apply whitespace suggestions from code review

These suggestions should make no semantic difference.

Co-authored-by: Bastian Köcher <[email protected]>

* Apply semantic from code review

These changes affect the semantics of the code; I'll follow up by ensuring that everything still works.

Co-authored-by: Bastian Köcher <[email protected]>

* add documentation to ValidationFunction type

* removing panicing private fn validation_function_params()

* expect validation function params to be in inherent data

* move OnValidationFunctionParams to primitives

* resolve weird formatting

* move mod validation_function_params into its own file

* add license to new file

Co-authored-by: Ricardo Rius <[email protected]>
Co-authored-by: Ricardo Rius <[email protected]>
Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Initial commit

Forked at: bf47cfa4
Parent branch: origin/master

* Update polkadot_chainspec.json

* Start message broker implementation

* Finish first stub implementation

* Add features

* Fix attribute

* Update primitives/src/lib.rs

Co-Authored-By: Joshy Orndorff <[email protected]>

Co-authored-by: Joshy Orndorff <[email protected]>

* Start fixing compilation errors

* Switch to git version of `WasmBuilder`

* Fix compilation

* More updates

* Adapt to latest Substrate/Polkadot changes

* Adds some highlevel docs for Cumulus

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <[email protected]>
Co-Authored-By: Joshy Orndorff <[email protected]>
Co-Authored-By: Nikolay Volf <[email protected]>

* Update docs/overview.md

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update docs/overview.md

Co-Authored-By: Robert Habermeier <[email protected]>

* Add sentence

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: Nikolay Volf <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>

This action will publish all pre-released, edited and published releases to a dedicated release notes Matrix room using https://github.com/marketplace/actions/matrix-message

@s3krit already added the necessary secrets to the repository settings.

Fixes #56 and #44

* Update Polkadot/Substrate

* Update branch

* add polkadot build script

* Add scripting to bring up a simple alice-bob example net

Demonstrated to produce blocks, but as of right now there's still
trouble getting it to respond to external queries on its ports.

* enable external rpc access to the nodes

Also shrink the build context by excluding some extraneous data.

* Ensure external RPC access works

Also set default branch appropriately, and have the stop command
clean itself up more thoroughly.

* Add multi-stage dockerfile for building the cumulus-test-parachain-collator

- Exclude the docker/ directory from build context because we're
  never going to build recursively, and this prevents spurious
  cache misses
- build the parachain collator in three stages. The build stage
  is discarded; the collator stage has a wrapper script to simplify
  generating the right bootnodes flags, and the default stage
  has just the binary in a small runtime.
- build_collator.sh collects appropriate build flags for the dockerfile
- inject_bootnodes.sh discovers the testnet node IDs and inserts them
  into the arguments list for cumulus-test-parachain-collator

* Add services which generate genesis state, run the collator

- Ignore the scripts directory to reduce spurious cache misses.
- Move inject_bootnodes.sh from the scripts directory into the root:
  It can't stay in the scripts directory, because that's ignored;
  I didn't want to invent _another_ top-level subdirectory for it.
  That decision could certainly be appealed, though.
- Move docker-compose.yml, add dc.sh, modify *_collator.sh: by
  taking docker-compose.yml out of the root directory, we can
  further reduce cache misses. However, docker-compose normally
  has a strong expectation that docker-compose.yml exist in the
  project root; it takes a moderately complicated invocation to
  override that expectation. That override is encoded in dc.sh;
  the updates to the other scripts are just to use the override.

The expectation as of now is that scripts/run_collator.sh runs
both chain nodes and the collator, generates the genesis state
into a volume with a transient container, and runs the collator
as specified in the repo README.

Upcoming work: Steps 5 and 6 from the readme.

* Launch the collator node

The biggest change here is adding the testing_net network to the
collator node's networks list. This lets it successfully connect
to the alice and bob nodes, which in turn lets it get their node IDs,
which was the blocker for a long time.

Remove httpie in favor of curl: makes for a smaller docker image,
and has fewer weird failure modes within docker.

Unfortunately this doesn't yet actually connect to the relay chain
nodes; that's the next area to figure out.

* enable external websocket access to indexer nodes

* Reorganize for improved caching, again

- Manually enumerate the set of source directories to copy when building.
  This bloats the cache a bit, but means that rebuilds on script changes
  don't bust that cache, which saves a _lot_ of time.
- Un-.dockerignore the scripts directory; it's small and will no longer
  trigger cache misses.
- Move inject_bootnodes.sh back into scripts directory for better organization.
- inject_bootnodes.sh: use rpc port for rpc call and p2p port for
  generating the bootnode string. I'm not 100% sure this is correct,
  but upwards of 80% at least.
- docker-compose.yml: reorganize the launch commands such that alice
  and bob still present the same external port mapping to the world,
  but within the docker-compose network, they both use the same
  (standard) p2p, rpc, and websocket ports. This makes life easier
  for inject_bootnodes.sh

The collator node still doesn't actually connect, but I think this
commit still represents real progress in that direction.

* Get the collator talking to the indexer nodes

In the end, it was four characters: -- and two = signs in the
launch arguments. They turn out to be critical characters for
correct operation, though!

Next up: automating step 5.

* Add runtime stage to collect runtime wasm blob into volume

We can't just copy the blob in the builder stage because the volumes
aren't available at that point.

Rewrite build_collator.sh into build_docker.sh and update for generality.

* WIP: add registrar service and partial work to actually register the collator

This is likely to be discarded; the Python library in use is 3rd party
and not well documented, while the official polkadot-js repo has a
CLI tool: https://github.com/polkadot-js/tools/tree/master/packages/api-cli

* Add a parachain registrar which should properly register the parachain

Doesn't work at the moment because it depends on two api-cli features
which I added today, which have not yet made it out into a published
release.

Next up: figure out how to add the `api-cli` at its `master` branch,
then run tests to ensure the collator is producing blocks. Then,
automate the block production tests.

* BROKEN attempt to demo registrar communication with the blockchain

This is a really weird bug. After running `scripts/run_collector.sh`,
which brings everything up, it's perfectly possible to get into
a state very much like what the registrar is in, and communicate
with the blockchain without issue:

```sh
$ docker run --rm --net cumulus_testing_net para-reg:latest polkadot-js-api --ws ws://172.28.1.1:9944 query.sudo.key
Thu 20 Feb 2020 12:19:20 PM CET
{
  "key": "5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY"
}
```

However, the registrar itself, doing the same thing from within
`register_para.sh`, is failing to find the right place in the network:

```
/runtime/cumulus_test_parachain_runtime.compact.wasm found after 0 seconds
/genesis/genesis-state found after 0 seconds
2020-02-20 10:43:22          API-WS: disconnected from ws://172.28.1.1:9944

 code: '1006' reason: 'connection failed'
_Event {
  type: 'error',
  isTrusted: false,
  _yaeti: true,
  target: W3CWebSocket {
    _listeners: {},
    addEventListener: [Function: _addEventListener],
    removeEventListener: [Function: _removeEventListener],
    dispatchEvent: [Function: _dispatchEvent],
    _url: 'ws://172.28.1.1:9944',
    _readyState: 3,
    _protocol: undefined,
    _extensions: '',
    _bufferedAmount: 0,
    _binaryType: 'arraybuffer',
    _connection: undefined,
    _client: WebSocketClient {
      _events: [Object: null prototype] {},
      _eventsCount: 0,
      _maxListeners: undefined,
      config: [Object],
      _req: null,
      protocols: [],
      origin: undefined,
      url: [Url],
      secure: false,
      base64nonce: 'aJ6J3pYDz8l5owVWHGbzHg==',
      [Symbol(kCapture)]: false
    },
    onclose: [Function (anonymous)],
    onerror: [Function (anonymous)],
    onmessage: [Function (anonymous)],
    onopen: [Function (anonymous)]
  },
  cancelable: true,
  stopImmediatePropagation: [Function (anonymous)]
}
```

They should be connected to the same network, running the same
image, doing the same call. The only difference is the file
existence checks, which really shouldn't be affecting the network
state at all.

Pushing this commit to ask for outside opinions on it, because
this is very weird and I clearly don't understand some part of
what's happening.

* Fix broken parachain registrar

The problem was that the registrar container was coming up too fast,
so the Alice node wasn't yet ready to receive connections. Using
a well-known wait script fixes the issue.

Next up: verify that the collator is in fact building blocks.

* fixes which cause the collator to correctly produce new parachain blocks

It didn't take much! The biggest issue was that the genesis state
was previously being double-encoded.

* add documentation for running the parachain automatically

* Add health check to collator

* minor scripting improvements

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

* Docker: copy the whole workspace in one go

Pro: future-proofing against the time we add or remove a directory
Con: changing any file in the workspace busts Rust's build cache,
     which takes a long time.

Co-authored-by: Bastian Köcher <[email protected]>

Otherwise the first positional argument is just swallowed

Fixes #62

Forked at: e81bbaae
Parent branch: origin/master