update provisioner subsystem (#1257)

* update provisioner subsystem

Closes #1143

* update with answers to the questions posed by previous todos

* add misbehavior reports, disputes to provisioner messages

* expand on the protocol

* updates per code review

polkadot/roadmap/implementors-guide/src/node/backing/candidate-backing.md

@@ -6,7 +6,7 @@ Its role is to produce backable candidates for inclusion in new relay-chain bloc
 
 Note that though the candidate backing subsystem attempts to produce as many backable candidates as possible, it does _not_ attempt to choose a single authoritative one. The choice of which actually gets included is ultimately up to the block author, by whatever metrics it may use; those are opaque to this subsystem.
 
-Once a sufficient quorum has agreed that a candidate is valid, this subsystem notifies the [Overseer](/node/overseer.html), which in turn engages block production mechanisms to include the parablock.
+Once a sufficient quorum has agreed that a candidate is valid, this subsystem notifies the [Provisioner](/node/utility/provisioner.html), which in turn engages block production mechanisms to include the parablock.
 
 ## Protocol
 

polkadot/roadmap/implementors-guide/src/node/utility/provisioner.md

 # Provisioner
 
-This subsystem is responsible for providing data to an external block authorship service beyond the scope of the [Overseer](/node/overseer.html) so that the block authorship service can author blocks containing data produced by various subsystems.
+Relay chain block authorship authority is governed by BABE and is beyond the scope of the Overseer and the rest of the subsystems. That said, ultimately the block author needs to select a set of backable parachain candidates and other consensus data, and assemble a block from them. This subsystem is responsible for providing the necessary data to all potential block authors.
 
-In particular, the data to provide:
+A major feature of the provisioner: this subsystem is responsible for ensuring that parachain block candidates are sufficiently available before sending them to potential block authors.
 
-- backable candidates and their backings
-- signed bitfields
-- misbehavior reports
-- dispute inherent
-    > TODO: needs fleshing out in validity module, related to blacklisting
+## Provisionable Data
+
+There are several distinct types of provisionable data, but they share this property in common: all should eventually be included in a relay chain block.
+
+### Backed Candidates
+
+The block author can choose 0 or 1 backed parachain candidates per parachain; the only constraint is that each backed candidate has the appropriate relay parent. However, the choice of a backed candidate must be the block author's; the provisioner must ensure that block authors are aware of all available [`BackedCandidate`s](/type-definitions.html#backed-candidate).
+
+### Signed Bitfields
+
+[Signed bitfields](/type-definitions.html#signed-availability-bitfield) are attestations from a particular validator about which candidates it believes are available.
+
+### Misbehavior Reports
+
+Misbehavior reports are self-contained proofs of misbehavior by a validator or group of validators. For example, it is very easy to verify a double-voting misbehavior report: the report contains two votes signed by the same key, advocating different outcomes. Concretely, misbehavior reports become inherents which cause dots to be slashed.
+
+Note that there is no mechanism in place which forces a block author to include a misbehavior report which it doesn't like, for example if it would be slashed by such a report. The chain's defense against this is to have a relatively long slash period, such that it's likely to encounter an honest author before the slash period expires.
+
+### Dispute Inherent
+
+The dispute inherent is similar to a misbehavior report in that it is an attestation of misbehavior on the part of a validator or group of validators. Unlike a misbehavior report, it is not self-contained: resolution requires coordinated action by several validators. The canonical example of a dispute inherent involves an approval checker discovering that a set of validators has improperly approved an invalid parachain block: resolving this requires the entire validator set to re-validate the block, so that the minority can be slashed.
+
+Dispute resolution is complex and is explained in substantially more detail [here](/runtime/validity.html).
+
+> TODO: The provisioner is responsible for selecting remote disputes to replay. Let's figure out the details.
 
 ## Protocol
 
-Input:
+Input: [`ProvisionerMessage`](/type-definitions.html#provisioner-message). Backed candidates come from the [Candidate Backing subsystem](/node/backing/candidate-backing.html), signed bitfields come from the [Bitfield Distribution subsystem](/node/availability/bitfield-distribution.html), and misbehavior reports and disputes come from the [Misbehavior Arbitration subsystem](/node/utility/misbehavior-arbitration.html).
 
-- Bitfield(relay_parent, signed_bitfield)
-- BackableCandidate(relay_parent, candidate_receipt, backing)
-- RequestBlockAuthorshipData(relay_parent, response_channel)
+At initialization, this subsystem has no outputs. Block authors can send a `ProvisionerMessage::RequestBlockAuthorshipData`, which includes a channel over which provisionable data can be sent. All appropriate provisionable data will then be sent over this channel, as it is received.
+
+Note that block authors must re-send a `ProvisionerMessage::RequestBlockAuthorshipData` for each relay parent they are interested in receiving provisionable data for.
 
 ## Functionality
 
-Use `StartWork` and `StopWork` to manage a set of jobs for relay-parents we might be building upon.
-Forward all messages to corresponding job, if any.
+The subsystem should maintain a set of handles to Block Authorship Provisioning Jobs that are currently live.
 
-## Block Authorship Provisioning Job
+### On Overseer Signal
 
-Track all signed bitfields, all backable candidates received. Provide them to the `RequestBlockAuthorshipData` requester via the `response_channel`. If more than one backable candidate exists for a given `Para`, provide the first one received.
+- `StartWork`: spawn a Block Authorship Provisioning Job with the given relay parent, storing a bidirectional channel with that job.
+- `StopWork`: terminate the Block Authorship Provisioning Job for the given relay parent, if any.
+
+### On `ProvisionerMessage`
+
+Forward the message to the appropriate Block Authorship Provisioning Job, or discard if no appropriate job is currently active.
+
+## Block Authorship Provisioning Job
 
-> TODO: better candidate-choice rules.
+Maintain the set of channels to block authors. On receiving provisionable data, send a copy over each channel.

polkadot/roadmap/implementors-guide/src/type-definitions.md

@@ -156,7 +156,7 @@ enum StatementDistributionMessage {
 ## Misbehavior Arbitration Message
 
 ```rust
-enum MisbehaviorArbitrationMessage {
+enum MisbehaviorReport {
   /// These validator nodes disagree on this candidate's validity, please figure it out
   ///
   /// Most likely, the list of statments all agree except for the final one. That's not
@@ -173,6 +173,33 @@ enum MisbehaviorArbitrationMessage {
 }
 ```
 
+## Provisioner Message
+
+```rust
+/// This data becomes intrinsics or extrinsics which should be included in a future relay chain block.
+enum ProvisionableData {
+  /// This bitfield indicates the availability of various candidate blocks.
+  Bitfield(Hash, SignedAvailabilityBitfield),
+  /// The Candidate Backing subsystem believes that this candidate is valid, pending availability.
+  BackedCandidate(BackedCandidate),
+  /// Misbehavior reports are self-contained proofs of validator misbehavior.
+  MisbehaviorReport(Hash, MisbehaviorReport),
+  /// Disputes trigger a broad dispute resolution process.
+  Dispute(Hash, Signature),
+}
+
+/// Message to the Provisioner.
+///
+/// In all cases, the Hash is that of the relay parent.
+enum ProvisionerMessage {
+  /// This message allows potential block authors to be kept updated with all new authorship data
+  /// as it becomes available.
+  RequestBlockAuthorshipData(Hash, Sender<ProvisionableData>),
+  /// This data should become part of a relay chain block
+  ProvisionableData(ProvisionableData),
+}
+```
+
 ## Host Configuration
 
 The internal-to-runtime configuration of the parachain host. This is expected to be altered only by governance procedures.