Disputes High-level rewrite & Disputes runtime (#2424)

* REVERT: comment out graphviz

* rewrite most of protocol-disputes

* write about conclusion and  chain selection

* tie back in overview

* basic disputes module

* guide: InclusionInherent -> ParaInherent

* language

* add ParaInherentData type

* plug parainherentdata into provisioner

* provide_multi_dispute

* tweak

* inclusion pipeline logic for disputes

* be clearer about signature checking

* reject backing of disputed blocks

* some type rejigging

* known-disputes runtime API

* wire up inclusion

* Revert "REVERT: comment out graphviz"

This reverts commit 66203e36.

* timeouts

* include in initialization order

* address grumbles

node/subsystem/src/messages.rs

@@ -543,7 +543,7 @@ pub enum ProvisionableData {
 
 /// This data needs to make its way from the provisioner into the InherentData.
 ///
-/// There, it is used to construct the InclusionInherent.
+/// There, it is used to construct the ParaInherent.
 pub type ProvisionerInherentData = (Vec<SignedAvailabilityBitfield>, Vec<BackedCandidate>);
 
 /// Message to the Provisioner.
@@ -558,7 +558,7 @@ pub enum ProvisionerMessage {
 	/// associated with a particular potential block hash.
 	///
 	/// This is expected to be used by a proposer, to inject that information into the InherentData
-	/// where it can be assembled into the InclusionInherent.
+	/// where it can be assembled into the ParaInherent.
 	RequestInherentData(Hash, oneshot::Sender<ProvisionerInherentData>),
 	/// This data should become part of a relay chain block
 	ProvisionableData(Hash, ProvisionableData),

roadmap/implementers-guide/src/SUMMARY.md

@@ -16,7 +16,7 @@
   - [Paras Module](runtime/paras.md)
   - [Scheduler Module](runtime/scheduler.md)
   - [Inclusion Module](runtime/inclusion.md)
-  - [InclusionInherent Module](runtime/inclusioninherent.md)
+  - [ParaInherent Module](runtime/parainherent.md)
   - [DMP Module](runtime/dmp.md)
   - [UMP Module](runtime/ump.md)
   - [HRMP Module](runtime/hrmp.md)
@@ -30,6 +30,7 @@
   - [Validation Code](runtime-api/validation-code.md)
   - [Candidate Pending Availability](runtime-api/candidate-pending-availability.md)
   - [Candidate Events](runtime-api/candidate-events.md)
+  - [Known Disputes](runtime-api/known-disputes.md)
 - [Node Architecture](node/README.md)
   - [Subsystems and Jobs](node/subsystems-and-jobs.md)
   - [Overseer](node/overseer.md)
@@ -70,6 +71,7 @@
     - [Messages](types/messages.md)
     - [Network](types/network.md)
     - [Approvals](types/approval.md)
+    - [Disputes](types/disputes.md)
 
 [Glossary](glossary.md)
 [Further Reading](further-reading.md)

roadmap/implementers-guide/src/node/utility/provisioner.md

@@ -40,7 +40,7 @@ Note that block authors must re-send a `ProvisionerMessage::RequestBlockAuthorsh
 
 ## Block Production
 
-When a validator is selected by BABE to author a block, it becomes a block producer. The provisioner is the subsystem best suited to choosing which specific backed candidates and availability bitfields should be assembled into the block. To engage this functionality, a `ProvisionerMessage::RequestInherentData` is sent; the response is a set of non-conflicting candidates and the appropriate bitfields. Non-conflicting means that there are never two distinct parachain candidates included for the same parachain and that new parachain candidates cannot be backed until the previous one either gets declared available or expired.
+When a validator is selected by BABE to author a block, it becomes a block producer. The provisioner is the subsystem best suited to choosing which specific backed candidates and availability bitfields should be assembled into the block. To engage this functionality, a `ProvisionerMessage::RequestInherentData` is sent; the response is a [`ParaInherentData`](../../types/runtime.md#parainherentdata). There are never two distinct parachain candidates included for the same parachain and that new parachain candidates cannot be backed until the previous one either gets declared available or expired. Appropriate bitfields, as outlined in the section on [bitfield selection](#bitfield-selection), and any dispute statements should be attached as well.
 
 ### Bitfield Selection
 

roadmap/implementers-guide/src/protocol-disputes.md

@@ -2,73 +2,67 @@
 
 Fast forward to [more detailed disputes requirments](./disputes-flow.md).
 
-## Motivation
+## Motivation and Background
 
-All blocks that end up on chain should be valid.
+All parachain blocks that end up in the finalized relay chain should be valid. This does not apply to blocks that are only backed, but not included.
 
-To ensure attempts, successful or not, of including
-a block that is invalid in respect to the validation code, must therefore be handled in some way with the offenders being punished and the whistle blowers being rewarded. Disputes and their resolution are the formal process to resolve these situations.
+We have two primary components for ensuring that nothing invalid ends up in the finalized relay chain:
+  * Approval Checking, as described [here](./protocol-approval.md) and implemented according to the [Approval Voting](node/approval/approval-voting.md) subsystem. This protocol can be shown to prevent invalid parachain blocks from making their way into the finalized relay chain as long as the amount of attempts are limited.
+  * Disputes, this protocol, which ensures that each attempt to include something bad is caught, and the offending validators are punished.
 
-At some point a validator claims that the `PoV` (proof of validity) - which was distributed with candidate block - for a certain block is invalid.
+Disputes and their resolution are the formal process to resolve these situations.
 
-Now the dispute can be happening quite some time later than the inclusion, but also right during backing. As such the block is stored or more so its Reed-Solomon encoded erasure chunks, from which the
-PoV can be reconstructed.
+Every dispute stems from a disagreement among two or more validators. If a bad actor creates a bad block, but the bad actor never distributes it to honest validators, then nobody will dispute it. Of course, such a situation is not even an attack on the network, so we don't need to worry about defending against it.
 
-A reconstructed PoV can be verified with the defined verification code, that is valid during the session the block was included or backed.
-If the block is invalid and there exists at least one backing vote and one validity challenging vote, a dispute exists.
-The existence of a dispute is detected by a backing checker
-or, if the block made it through backing stage, by an approval checker.
-In either case, the validator casts and distributes its vote via means of gossip.
+From most to least important, here are the attack scenarios we are interested in identifying and deterring:
+  * A parablock included on a branch of the relay chain is bad
+  * A parablock backed on a branch of the relay chain is bad
+  * A parablock seconded, but not backed on any branch of the relay chain, is bad.
 
-At this point the set of backing validators can not be trusted (since they voted for the block despite something being
-fishy at the very least). On the other hand, one must also consider, the validator that blew the whistle has ulterior motives
-to do so (i.e. it is controlled by a third party and wants to incur damage to itself).
-In either way, there are malicious validators around.
-As a consequence, all validators at the time of block backing, are being notified via broadcast of
-the first pair of backing and challenging vote.
-Validators that backed the candidate implicitly count as votes. Those validators are allowed to cast
-a regular vote (a non-backing vote) as well, but it is generally not in their interest to vote both sides, since that would
-advance the progress towards supermajority either way and have their bonds slashed.
-If both votes lean in the same direction, i.e. both positive they are only counted as one.
-Two opposing votes by the same validator would be equal to an attempted double vote and would be slashed accordingly.
+As covered in the [protocol overview](./protocol-overview.md), checking a parachain block requires 3 pieces of data: the parachain validation code, the [`AvailableData`](types/availability.md), and the [`CandidateReceipt`](types/candidate.md). The validation code is available on-chain, and published ahead of time, so that no two branches of the relay chain have diverging views of the validation code for a given parachain. Note that only for the first scenario, where the parablock has been included on a branch of the relay chain, is the data necessarily available. Thus, dispute processes should begin with an availability process to ensure availability of the `AvailableData`. This availability process will conclude quickly if the data is already available. If the data is not already available, then the initiator of the dispute must make it available.
 
-All validators at block inclusion time are eligible to (and should) cast their Vote. The backing votes of backing checkers
-are counted as votes as well.
+Disputes have both an on-chain and an off-chain component. Slashing and punishment is handled on-chain, so votes by validators on either side of the dispute must be placed on-chain. Furthermore, a dispute on one branch of the relay chain should be transposed to all other active branches of the relay chain. The fact that slashing occurs _in all histories_ is crucial for deterring attempts to attack the network. The attacker should not be able to escape with their funds because the network has moved on to another branch of the relay chain where no attack was attempted.
+
+In fact, this is why we introduce a distinction between _local_ and _remote_ disputes. We categorize disputes as either local or remote relative to any particular branch of the relay chain. Local disputes are about dealing with our first scenario, where a parablock has been included on the specific branch we are looking at. In these cases, the chain is corrupted all the way back to the point where the parablock was backed and must be discarded. However, as mentioned before, the dispute must propagate to all other branches of the relay chain. All other disputes are considered _remote_. For the on-chain component, when handling a dispute for a block which was not included in the current fork of the relay chain, it is impossible to discern between our attack scenarios. It is possible that the parablock was included somewhere, or backed somewhere, or wasn't backed anywhere. The on-chain component for handling these cases will be the same.
 
 ## Initiation
 
-A dispute is initiated by one approval checker creating and gossiping a vote, that challenges another vote.
+Disputes are initiated by any validator who finds their opinion on the validity of a parablock in opposition to another issued statement. As all statements currently gathered by the relay chain imply validity, disputes will be initiated only by nodes which perceive that the parablock is bad.
+
+The initiation of a dispute begins off-chain. A validator signs a message indicating that it disputes the validity of the parablock and notifies all other validators, off-chain, of all of the statements it is aware of for the disputed parablock. These may be backing statements or approval-checking statements. It is worth noting that there is no special message type for initiating a dispute. It is the same message as is used to participate in a dispute and vote negatively. As such, there is no consensus required on who initiated a dispute, only on the fact that there is a dispute in-progress.
+
+In practice, the initiator of a dispute will be either one of the backers or one of the approval checkers for the parablock. If the result of execution is found to be invalid, the validator will initiate the dispute as described above. Furthermore, if the dispute occurs during the backing phase, the initiator must make the data available to other validators. If the dispute occurs during approval checking, the data is already available.
+
+Lastly, it is possible that for backing disputes, i.e. where the data is not already available among all validators, that an adversary may DoS the few parties who are checking the block to prevent them from distributing the data to other validators participating in the dispute process. Note that this can only occur pre-inclusion for any given parablock, so the downside of this attack is small and it is not security-critical to address these cases. However, we assume that the adversary can only prevent the validator from issuing messages for a limited amount of time. We also assume that there is a side-channel where the relay chain's governance mechanisms can trigger disputes by providing the full PoV and candidate receipt on-chain manually.
+
+## Dispute Participation
+
+Once becoming aware of a dispute, it is the responsibility of all validators to participate in the dispute. Concretely, this means:
+  * Circulate all statements about the candidate that we are aware of - backing statements, approval checking statements, and dispute statements.
+  * If we have already issued any type of statement about the candidate, go no further.
+  * Download the [`AvailableData`](types/availability.md). If possible, this should first be attempted from other dispute participants or backing validators, and then [(via erasure-coding)](node/availability/availability-recovery.md) from all validators. 
+  * Extract the Validation Code from any recent relay chain block. Code is guaranteed to be kept available on-chain, so we don't need to download any particular fork of the chain.
+  * Execute the block under the validation code, using the `AvailableData`, and check that all outputs are correct, including the `erasure-root` of the [`CandidateReceipt`](types/candidate.md).
+  * Issue a dispute participation statement to the effect of the validity of the candidate block.
+
+Disputes _conclude_ after ⅔ supermajority is reached in either direction. 
 
-After a backing or approval checker challenged a block, all validators that received the gossiped vote, reconstruct the block
-from availability erasure code chunks and check the block's PoV themselves via the validation code.
-The result of that check is converted into a vote, and distributed via the same mechanics as the first one.
+The on-chain component of disputes can be initiated by providing any two conflicting votes and it also waits for a ⅔ supermajority on either side. The on-chain component also tracks which parablocks have already been disputed so the same parablock may only be disputed once on any particular branch of the relay chain. Lastly, it also tracks which blocks have been included on the current branch of the relay chain. When a dispute is initiated for a para, inclusion is halted for the para until the dispute concludes. 
 
-Once a receiver receives ⅔ supermajority in one or the other direction, the
-vote is concluded.
-Conclusion implies that the result for this block can not be altered anymore, valid or invalid is fixed now.
+The author of a relay chain block should initiate the on-chain component of disputes for all disputes which the chain is not aware of, and provide all statements to the on-chain component as well. This should all be done via _inherents_.
 
-In order to ensure, the dispute result is not forgotten or intentionally side stepped, it has to be recorded on chain.
-This on chain recording mechanic must be vigilant, in a sense, that new emerging forks
-must also receive the dispute resolution recorded (transplantation) irrespectively of the disputed block being included in that chain or not.
+Validators can learn about dispute statements in two ways:
+  * Receiving them from other validators over gossip
+  * Scraping them from imported blocks of the relay chain. This is also used for validators to track other types of statements, such as backing statements.
 
-If the disputed block was already finalized, the chain must be put in governance mode to be resolved by human interaction
-(i.e. sudo or motion or other mechanics that are available ).
+Validators are rewarded for providing statements to the chain as well as for participating in the dispute, on either side. However, the losing side of the dispute is slashed.
 
-As such the validator has to keep track of all votes irrespective if the disputed block is already known or not.
-All backing votes should be either kept in storage as well, or be queried on demand, since they are a kind of vote
-as well.
+## Dispute Conclusion
 
-## Late votes
+Disputes, roughly, are over when one side reaches a ⅔ supermajority. They may also conclude after a timeout, without either side witnessing supermajority, which will only happen if the majority of validators are unable to vote for some reason. Furthermore, disputes on-chain will stay open for some fixed amount of time even after concluding, to accept new votes.
 
 Late votes, after the dispute already reached a ⅔ supermajority, must be rewarded (albeit a smaller amount) as well.
-These ones must be attached to the votes after a defined period of time after the result has reached
-the required ⅔ supermajority.
 
 ## Chain Selection / Grandpa
 
-Chain selection should be influenced by the chance of picking a chain that does not even include the disputed block.
-Hence removing the need to include the dispute resolution itself.
-This is only possible though, if the set of active heads contains such a fork.
-In Grandpa the Voting rule should be used to avoid finalizing chains that contain an open or negative shut (shut with supermajority that marks the block as invalid) dispute.
-In case all possible chains contains such a dispute, a metric must be used to decide which fork to use or avoid finalization until one dispute resolves positive (the
-block is valid).
+The [Approval Checking](protocol-approval.md) protocol prevents finalization of chains that contain parablocks that are not yet approved. With disputes, we take it one step further and do not vote to finalize any chains which contain parablocks that are being disputed or have lost a dispute anywhere.

roadmap/implementers-guide/src/protocol-overview.md

@@ -41,10 +41,10 @@ The Approval Process, at a glance, looks like this:
 1. Parablocks that have been included by the Inclusion Pipeline are pending approval for a time-window known as the secondary checking window.
 1. During the secondary-checking window, validators randomly self-select to perform secondary checks on the parablock.
 1. These validators, known in this context as secondary checkers, acquire the parablock and its PoV, and re-run the validation function.
-1. The secondary checkers submit the result of their checks to the relay chain. Contradictory results lead to escalation, where all validators are required to check the block. The validators on the losing side of the dispute are slashed.
+1. The secondary checkers gossip the result of their checks. Contradictory results lead to escalation, where all validators are required to check the block. The validators on the losing side of the dispute are slashed.
 1. At the end of the Approval Process, the parablock is either Approved or it is rejected. More on the rejection process later.
 
-More information on the Approval Process can be found in the dedicated section on [Approval](protocol-approval.md).
+More information on the Approval Process can be found in the dedicated section on [Approval](protocol-approval.md). More information on Disputes can be found in the dedicated section on [Disputes](protocol-disputes.md).
 
 These two pipelines sum up the sequence of events necessary to extend and acquire full security on a Parablock. Note that the Inclusion Pipeline must conclude for a specific parachain before a new block can be accepted on that parachain. After inclusion, the Approval Process kicks off, and can be running for many parachain blocks at once.
 

roadmap/implementers-guide/src/runtime-api/known-disputes.md

+# Known Disputes
+
+Get all disputes known by the chain. These disputes may be either live or concluded. The [`DisputeState`](../types/disputes.md#disputestate) can be used to determine whether the dispute still accepts votes, as well as which validators' votes may be included.
+
+```rust
+fn known_disputes() -> Vec<(SessionIndex, CandidateHash, DisputeState)>;
+```

roadmap/implementers-guide/src/runtime/disputes.md

@@ -5,102 +5,97 @@ After a backed candidate is made available, it is included and proceeds into an
 However, this isn't the end of the story. We are working in a forkful blockchain environment, which carries three important considerations:
 
 1. For security, validators that misbehave shouldn't only be slashed on one fork, but on all possible forks. Validators that misbehave shouldn't be able to create a new fork of the chain when caught and get away with their misbehavior.
-1. It is possible that the parablock being contested has not appeared on all forks.
+1. It is possible (and likely) that the parablock being contested has not appeared on all forks.
 1. If a block author believes that there is a disputed parablock on a specific fork that will resolve to a reversion of the fork, that block author is better incentivized to build on a different fork which does not include that parablock.
 
 This means that in all likelihood, there is the possibility of disputes that are started on one fork of the relay chain, and as soon as the dispute resolution process starts to indicate that the parablock is indeed invalid, that fork of the relay chain will be abandoned and the dispute will never be fully resolved on that chain.
 
 Even if this doesn't happen, there is the possibility that there are two disputes underway, and one resolves leading to a reversion of the chain before the other has concluded. In this case we want to both transplant the concluded dispute onto other forks of the chain as well as the unconcluded dispute.
 
-We account for these requirements by having the validity module handle two kinds of disputes.
+We account for these requirements by having the disputes module handle two kinds of disputes.
 
 1. Local disputes: those contesting the validity of the current fork by disputing a parablock included within it.
 1. Remote disputes: a dispute that has partially or fully resolved on another fork which is transplanted to the local fork for completion and eventual slashing.
 
-## Approval
+When a local dispute concludes negatively, the chain needs to be abandoned and reverted back to a block where the state does not contain the bad parablock. We expect that due to the [Approval Checking Protocol](../protocol-approval.md), the current executing block should not be finalized. So we do two things when a local dispute concludes negatively:
+1. Freeze the state of parachains so nothing further is backed or included.
+1. Issue a digest in the header of the block that signals to nodes that this branch of the chain is to be abandoned.
 
-We begin approval checks upon any candidate immediately once it becomes available.  
+If, as is expected, the chain is unfinalized, the freeze will have no effect as no honest validator will attempt to build on the frozen chain. However, if the approval checking protocol has failed and the bad parablock is finalized, the freeze serves to put the chain into a governance-only mode.
 
-Assigning approval checks involve VRF secret keys held by every validator, making it primarily an off-chain process.  All assignment criteria require specific data called "stories" about the relay chain block in which the candidate assigned by that criteria became available.  Among these criteria, the BABE VRF output provides the story for two, and the other's story consists of the candidate's block hash plus external knowledge that a relay chain equivocation exists with a conflicting candidate. 
+The storage of this module is designed around tracking [`DisputeState`s](../types/disputes.md#disputestate), updating them with votes, and tracking blocks included by this branch of the relay chain. It also contains a `Frozen` parameter designed to freeze the state of all parachains.
 
-We liberate availability cores when their candidate becomes available of course, but one approval assignment criteria continues associating each candidate with the core number it occupied when it became available. 
+## Storage
 
-Assignment proceeds in loosely timed rounds called `DelayTranche`s roughly 12 times faster than block production, in which validators send assignment notices until all candidates have enough checkers assigned.  Assignment tracks when approval votes arrive too and assigns more checkers if some checkers run late.
+Storage Layout:
 
-Approval checks provide more security than backing checks, so polkadot becomes more efficient when validators perform more approval checks per backing check.  If validators run 4 approval checks for every backing check, and run almost one backing check per relay chain block, then validators actually check almost 6 blocks per relay chain block.
+```rust
+LastPrunedSession: Option<SessionIndex>,
 
-We should therefore reward approval checkers correctly because approval checks should actually represent our single largest workload.  It follows that both assignment notices and approval votes should be tracked on-chain.  
+// All ongoing or concluded disputes for the last several sessions.
+Disputes: double_map (SessionIndex, CandidateHash) -> Option<DisputeState>,
+// All included blocks on the chain, as well as the block number in this chain that
+// should be reverted back to if the candidate is disputed and determined to be invalid.
+Included: double_map (SessionIndex, CandidateHash) -> Option<BlockNumber>,
+// Whether the chain is frozen or not. Starts as `false`. When this is `true`,
+// the chain will not accept any new parachain blocks for backing or inclusion.
+// It can only be set back to `false` by governance intervention.
+Frozen: bool,
+```
 
-We might track the assignments and approvals together as pairs in a simple rewards system.  There are however two reasons to witness approvals on chain by tracking assignments and approvals on-chain, rewards and finality integration.
+Configuration:
 
-First, an approval that arrives too slowly prompts assigning extra "no show" replacement checkers.  Yet, we consider a block valid if the earlier checker completes their work, even if the extra checkers never quite finish, which complicates rewarding these extra checkers.  We could support more nuanced rewards for extra checkers if assignments are placed on-chain earlier.  Assignment delay tranches progress 12ish times faster than the relay chain, but no shows could still be witness by the relay chain because the no show delay takes longer than a relay chain slot. 
+```rust
+/// How many sessions before the current that disputes should be accepted for.
+DisputePeriod: SessionIndex;
+/// How long after conclusion to accept statements.
+PostConclusionAcceptancePeriod: BlockNumber;
+/// How long is takes for a dispute to conclude by time-out, if no supermajority is reached.
+ConclusionByTimeOutPeriod: BlockNumber;
+```
 
-Second, we know off-chain when the approval process completes based upon all gossiped assignment notices, not just the approving ones.  We need not-yet-approved assignment notices to appear on-chain if the chain should know about the validity of recently approved blocks.  Relay chain blocks become eligible for finality in GRANDPA only once all their included candidates pass approvals checks, meaning all assigned checkers either voted approve or else were declared "no show" and replaced by more assigned checkers.  A purely off-chain approvals scheme complicates GRANDPA with additional objections logic.  
+## Session Change
 
-Integration with GRANDPA appears simplest if we witness approvals in chain:  Aside from inherents for assignment notices and approval votes, we provide an "Approved" inherent by which a relay chain block declares a past relay chain block approved.  In other words, it trigger the on-chain approval counting logic in a relay chain block `R1` to rerun the assignment and approval tracker logic for some ancestor `R0`, which then declares `R0` approved.  In this case, we could integrate with GRANDPA by gossiping messages that list the descendent `R1`, but then map this into the approved ancestor `R0` for GRANDPA itself.
+1. If the current session is not greater than `dispute_period + 1`, nothing to do here.
+1. Set `pruning_target = current_session - dispute_period - 1`. We add the extra `1` because we want to keep things for `dispute_period` _full_ sessions. The stuff at the end of the most recent session has been around for ~0 sessions, not ~1.
+1. If `LastPrunedSession` is `None`, then set `LastPrunedSession` to `Some(pruning_target)` and return.
+1. Otherwise, clear out all disputes and included candidates in the range `last_pruned..=pruning_target` and set `LastPrunedSession` to `Some(pruning_target)`.
 
-Approval votes could be recorded on-chain quickly because they represent a major commitments.  
+## Block Initialization
 
-Assignment notices should be recorded on-chain only when relevant.  Any sent too early are retained but ignore until relevant by our off-chain assignment system.  Assignments are ignored completely by the dispute system because any dispute immediately escalates into all validators checking, but disputes count existing approval votes of course.
+1. Iterate through all disputes. If any have not concluded and started more than `ConclusionByTimeOutPeriod` blocks ago, set them to `Concluded` and mildly punish all validators associated, as they have failed to distribute available data.
 
+## Routines
 
-## Local Disputes
+* `provide_multi_dispute_data(MultiDisputeStatementSet) -> Vec<(SessionIndex, Hash)>`:
+  1. Fail if any disputes in the set are duplicate or concluded before the `PostConclusionAcceptancePeriod` window relative to now.
+  1. Pass on each dispute statement set to `provide_dispute_data`, propagating failure.
+  1. Return a list of all candidates who just had disputes initiated.
 
-There is little overlap between the approval system and the disputes systems since disputes cares only that two validators disagree.  We do however require that disputes count validity votes from elsewhere, both the backing votes and the approval votes.  
+* `provide_dispute_data(DisputeStatementSet) -> bool`: Provide data to an ongoing dispute or initiate a dispute.
+  1. All statements must be issued under the correct session for the correct candidate. 
+  1. `SessionInfo` is used to check statement signatures and this function should fail if any signatures are invalid.
+  1. If there is no dispute under `Disputes`, create a new `DisputeState` with blank bitfields.
+  1. If `concluded_at` is `Some`, and is `concluded_at + PostConclusionAcceptancePeriod < now`, return false.
+  1. Import all statements into the dispute. This should fail if any disputes are duplicate; if the corresponding bit for the corresponding validator is set in the dispute already.
+  1. If `concluded_at` is `None`, reward all statements slightly less.
+  1. If `concluded_at` is `Some`, reward all statements slightly less.
+  1. If either side now has supermajority, slash the other side. This may be both sides, and we support this possibility in code, but note that this requires validators to participate on both sides which has negative expected value. Set `concluded_at` to `Some(now)`.
+  1. If just concluded against the candidate and the `Included` map contains `(session, candidate)`: invoke `revert_and_freeze` with the stored block number.
+  1. Return true if just initiated, false otherwise.
 
-We could approve, and even finalize, a relay chain block which then later disputes due to claims of some parachain being invalid.
+* `disputes() -> Vec<(SessionIndex, CandidateHash, DisputeState)>`: Get a list of all disputes and info about dispute state.
+  1. Iterate over all disputes in `Disputes`. Set the flag according to `concluded`.
 
-> TODO: store all included candidate and attestations on them here. accept additional backing after the fact. accept reports based on VRF. candidate included in session S should only be reported on by validator keys from session S. trigger slashing. probably only slash for session S even if the report was submitted in session S+k because it is hard to unify identity
+* `note_included(SessionIndex, CandidateHash, included_in: BlockNumber)`:
+  1. Add `(SessionIndex, CandidateHash)` to the `Included` map with `included_in - 1` as the value.
+  1. If there is a dispute under `(SessionIndex, CandidateHash)` that has concluded against the candidate, invoke `revert_and_freeze` with the stored block number.
 
-One first question is to ask why different logic for local disputes is necessary. It seems that local disputes are necessary in order to create the first escalation that leads to block producers abandoning the chain and making remote disputes possible.
+* `could_be_invalid(SessionIndex, CandidateHash) -> bool`: Returns whether a candidate has a live dispute ongoing or a dispute which has already concluded in the negative.
 
-Local disputes are only allowed on parablocks that have been included on the local chain and are in the acceptance period.
+* `is_frozen()`: Load the value of `Frozen` from storage.
 
-For each such parablock, it is guaranteed by the inclusion pipeline that the parablock is available and the relevant validation code is available.
-
-Disputes may occur against blocks that have happened in the session prior to the current one, from the perspective of the chain. In this case, the prior validator set is responsible for handling the dispute and to do so with their keys from the last session. This means that validator duty actually extends 1 session beyond leaving the validator set.
-
-...
-
-After concluding with enough validtors voting, the dispute will remain open for some time in order to collect further evidence of misbehaving validators, and then issue a signal in the header-chain that this fork should be abandoned along with the hash of the last ancestor before inclusion, which the chain should be reverted to, along with information about the invalid block that should be used to blacklist it from being included.
-
-## Remote Disputes
-
-When a dispute has occurred on another fork, we need to transplant that dispute to every other fork. This poses some major challenges.
-
-There are two types of remote disputes. The first is a remote roll-up of a concluded dispute. These are simply all attestations for the block, those against it, and the result of all (secondary) approval checks. A concluded remote dispute can be resolved in a single transaction as it is an open-and-shut case of a quorum of validators disagreeing with another.
-
-The second type of remote dispute is the unconcluded dispute. An unconcluded remote dispute is started by any validator, using these things:
-
-- A candidate
-- The session that the candidate has appeared in.
-- Backing for that candidate
-- The validation code necessary for validation of the candidate.
-  > TODO: optimize by excluding in case where code appears in `Paras::CurrentCode` of this fork of relay-chain
-- Secondary checks already done on that candidate, containing one or more disputes by validators. None of the disputes are required to have appeared on other chains.
-  > TODO: validator-dispute could be instead replaced by a fisherman w/ bond
-
-When beginning a remote dispute, at least one escalation by a validator is required, but this validator may be malicious and desires to be slashed. There is no guarantee that the para is registered on this fork of the relay chain or that the para was considered available on any fork of the relay chain.
-
-So the first step is to have the remote dispute proceed through an availability process similar to the one in the [Inclusion Module](inclusion.md), but without worrying about core assignments or compactness in bitfields.
-
-We assume that remote disputes are with respect to the same validator set as on the current fork, as BABE and GRANDPA assure that forks are never long enough to diverge in validator set.
-> TODO: this is at least directionally correct. handling disputes on other validator sets seems useless anyway as they wouldn't be bonded.
-
-As with local disputes, the validators of the session the candidate was included on another chain are responsible for resolving the dispute and determining availability of the candidate.
-
-If the candidate was not made available on another fork of the relay chain, the availability process will time out and the disputing validator will be slashed on this fork. The escalation used by the validator(s) can be replayed onto other forks to lead the wrongly-escalating validator(s) to be slashed on all other forks as well. We assume that the adversary cannot censor validators from seeing any particular forks indefinitely
-
-> TODO: set the availability timeout for this accordingly - unlike in the inclusion pipeline we are slashing for unavailability here!
-
-If the availability process passes, the remote dispute is ready to be included on this chain. As with the local dispute, validators self-select based on a VRF. Given that a remote dispute is likely to be replayed across multiple forks, it is important to choose a VRF in a way that all forks processing the remote dispute will have the same one. Choosing the VRF is important as it should not allow an adversary to have control over who will be selected as a secondary approval checker.
-
-After enough validator self-select, under the same escalation rules as for local disputes, the Remote dispute will conclude, slashing all those on the wrong side of the dispute. After concluding, the remote dispute remains open for a set amount of blocks to accept any further proof of additional validators being on the wrong side.
-
-## Slashing and Incentivization
-
-The goal of the dispute is to garner a `>2/3` (`2f + 1`) supermajority either in favor of or against the candidate.
-
-For remote disputes, it is possible that the parablock disputed has never actually passed any availability process on any chain. In this case, validators will not be able to obtain the PoV of the parablock and there will be relatively few votes. We want to disincentivize voters claiming validity of the block from preventing it from becoming available, so we charge them a small distraction fee for wasting the others' time if the dispute does not garner a 2/3+ supermajority on either side. This fee can take the form of a small slash or a reduction in rewards.
-
-When a supermajority is achieved for the dispute in either the valid or invalid direction, we will penalize non-voters either by issuing a small slash or reducing their rewards. We prevent censorship of the remaining validators by leaving the dispute open for some blocks after resolution in order to accept late votes.
+* `revert_and_freeze(BlockNumber):
+  1. If `is_frozen()` return.
+  1. issue a digest in the block header which indicates the chain is to be abandoned back to the stored block number.
+  1. Set `Frozen` to true.

roadmap/implementers-guide/src/runtime/inclusion.md

@@ -59,7 +59,7 @@ All failed checks should lead to an unrecoverable error making the block invalid
   1. apply each bit of bitfield to the corresponding pending candidate. looking up parathread cores using the `core_lookup`. Disregard bitfields that have a `1` bit for any free cores.
   1. For each applied bit of each availability-bitfield, set the bit for the validator in the `CandidatePendingAvailability`'s `availability_votes` bitfield. Track all candidates that now have >2/3 of bits set in their `availability_votes`. These candidates are now available and can be enacted.
   1. For all now-available candidates, invoke the `enact_candidate` routine with the candidate and relay-parent number.
-  1. Return a list of freed cores consisting of the cores where candidates have become available.
+  1. Return a list of `(CoreIndex, CandidateHash)` from freed cores consisting of the cores where candidates have become available.
 * `process_candidates(parent_storage_root, BackedCandidates, scheduled: Vec<CoreAssignment>, group_validators: Fn(GroupIndex) -> Option<Vec<ValidatorIndex>>)`:
   1. check that each candidate corresponds to a scheduled core and that they are ordered in the same order the cores appear in assignments in `scheduled`.
   1. check that `scheduled` is sorted ascending by `CoreIndex`, without duplicates.
@@ -89,7 +89,7 @@ All failed checks should lead to an unrecoverable error making the block invalid
 * `collect_pending`:
 
   ```rust
-    fn collect_pending(f: impl Fn(CoreIndex, BlockNumber) -> bool) -> Vec<u32> {
+    fn collect_pending(f: impl Fn(CoreIndex, BlockNumber) -> bool) -> Vec<CoreIndex> {
       // sweep through all paras pending availability. if the predicate returns true, when given the core index and
       // the block number the candidate has been pending availability since, then clean up the corresponding storage for that candidate and the commitments.
       // return a vector of cleaned-up core IDs.
@@ -98,3 +98,4 @@ All failed checks should lead to an unrecoverable error making the block invalid
 * `force_enact(ParaId)`: Forcibly enact the candidate with the given ID as though it had been deemed available by bitfields. Is a no-op if there is no candidate pending availability for this para-id. This should generally not be used but it is useful during execution of Runtime APIs, where the changes to the state are expected to be discarded directly after.
 * `candidate_pending_availability(ParaId) -> Option<CommittedCandidateReceipt>`: returns the `CommittedCandidateReceipt` pending availability for the para provided, if any.
 * `pending_availability(ParaId) -> Option<CandidatePendingAvailability>`: returns the metadata around the candidate pending availability for the para, if any.
+* `collect_disputed(disputed: Vec<CandidateHash>) -> Vec<CoreIndex>`: Sweeps through all paras pending availability. If the candidate hash is one of the disputed candidates, then clean up the corresponding storage for that candidate and the commitments. Return a vector of cleaned-up core IDs.

roadmap/implementers-guide/src/runtime/initializer.md

@@ -24,6 +24,7 @@ The other parachains modules are initialized in this order:
 1. Scheduler
 1. Inclusion
 1. SessionInfo
+1. Disputes
 1. DMP
 1. UMP
 1. HRMP

roadmap/implementers-guide/src/runtime/inclusioninherent.md → roadmap/implementers-guide/src/runtime/parainherent.md

-# InclusionInherent
+# ParaInherent
 
-This module is responsible for all the logic carried by the `Inclusion` entry-point. This entry-point is mandatory, in that it must be invoked exactly once within every block, and it is also "inherent", in that it is provided with no origin by the block author. The data within it carries its own authentication. If any of the steps within fails, the entry-point is considered as having failed and the block will be invalid.
+This module is responsible for providing all data given to the runtime by the block author to the various parachains modules. The entry-point is mandatory, in that it must be invoked exactly once within every block, and it is also "inherent", in that it is provided with no origin by the block author. The data within it carries its own authentication; i.e. the data takes the form of signed statements by validators. If any of the steps within fails, the entry-point is considered as having failed and the block will be invalid.
 
 This module does not have the same initialization/finalization concerns as the others, as it only requires that entry points be triggered after all modules have initialized and that finalization happens after entry points are triggered. Both of these are assumptions we have already made about the runtime's order of operations, so this module doesn't need to be initialized or finalized by the `Initializer`.
 
+There are a couple of important notes to the operations in this inherent as they relate to disputes.
+1. We don't accept bitfields or backed candidates if in "governance-only" mode from having a local dispute conclude on this fork.
+1. When disputes are initiated, we remove the block from pending availability. This allows us to roll back chains to the block before blocks are included as opposed to backing. It's important to do this before processing bitfields.
+1. `Inclusion::collect_disputed` is kind of expensive so it's important to gate this on whether there are actually any new disputes. Which should be never.
+1. And we don't accept parablocks that have open disputes or disputes that have concluded against the candidate. It's important to import dispute statements before backing, but this is already the case as disputes are imported before processing bitfields.
+
 ## Storage
 
 ```rust
@@ -16,13 +22,19 @@ Included: Option<()>,
 
 ## Entry Points
 
-* `inclusion`: This entry-point accepts three parameters: The relay-chain parent block header, [`Bitfields`](../types/availability.md#signed-availability-bitfield) and [`BackedCandidates`](../types/backing.md#backed-candidate).
+* `enter`: This entry-point accepts three parameters: The relay-chain parent block header, [`Bitfields`](../types/availability.md#signed-availability-bitfield) and [`BackedCandidates`](../types/backing.md#backed-candidate).
     1. Hash the parent header and make sure that it corresponds to the block hash of the parent (tracked by the `frame_system` FRAME module),
+    1. Invoke `Disputes::provide_multi_dispute_data`.
+    1. If `Disputes::is_frozen`, return and set `Included` to `Some(())`.
+    1. If there are any created disputes from the current session, invoke `Inclusion::collect_disputed` with the disputed candidates. Annotate each returned core with `FreedReason::Concluded`.
     1. The `Bitfields` are first forwarded to the `Inclusion::process_bitfields` routine, returning a set of freed cores. Provide a `Scheduler::core_para` as a core-lookup to the `process_bitfields` routine. Annotate each of these freed cores with `FreedReason::Concluded`.
-    1. If `Scheduler::availability_timeout_predicate` is `Some`, invoke `Inclusion::collect_pending` using it, and add timed-out cores to the free cores, annotated with `FreedReason::TimedOut`.
+    1. For each freed candidate from the `Inclusion::process_bitfields` call, invoke `Disputes::note_included(current_session, candidate)`.
+    1. If `Scheduler::availability_timeout_predicate` is `Some`, invoke `Inclusion::collect_pending` using it and annotate each of those freed cores with `FreedReason::TimedOut`.
+    1. Combine and sort the dispute-freed cores, the bitfield-freed cores, and the timed-out cores.
     1. Invoke `Scheduler::clear`
-    1. Invoke `Scheduler::schedule(freed, System::current_block())`
+    1. Invoke `Scheduler::schedule(freed_cores, System::current_block())`
     1. Extract `parent_storage_root` from the parent header,
+    1. If `Disputes::could_be_invalid(current_session, candidate)` is true for any of the `backed_candidates`, fail.
     1. Invoke the `Inclusion::process_candidates` routine with the parameters `(parent_storage_root, backed_candidates, Scheduler::scheduled(), Scheduler::group_validators)`.
     1. Call `Scheduler::occupied` using the return value of the `Inclusion::process_candidates` call above, first sorting the list of assigned core indices.
     1. Call the `Ump::process_pending_upward_messages` routine to execute all messages in upward dispatch queues.

roadmap/implementers-guide/src/types/backing.md

@@ -64,8 +64,6 @@ enum Statement {
   Seconded(CommittedCandidateReceipt),
   /// A statement about the validity of a candidate, based on candidate's hash.
   Valid(Hash),
-  /// A statement about the invalidity of a candidate.
-  Invalid(Hash),
 }
 
 /// A statement about the validity of a parachain candidate.
@@ -79,8 +77,6 @@ enum CompactStatement {
   Seconded(Hash),
   /// A statement about the validity of a candidate, based on candidate's hash.
   Valid(Hash),
-  /// A statement about the invalidity of a candidate.
-  Invalid(Hash),
 }
 ```
 

roadmap/implementers-guide/src/types/disputes.md

+# Disputes
+
+## DisputeStatementSet 
+
+```rust
+/// A set of statements about a specific candidate.
+struct DisputeStatementSet {
+    candidate_hash: CandidateHash,
+    session: SessionIndex,
+    statements: Vec<(DisputeStatement, ValidatorIndex, ValidatorSignature)>,
+}
+```
+
+## DisputeStatement
+
+```rust
+/// A statement about a candidate, to be used within some dispute resolution process.
+///
+/// Statements are either in favor of the candidate's validity or against it.
+enum DisputeStatement {
+    /// A valid statement, of the given kind
+    Valid(ValidDisputeStatementKind),
+    Invalid(InvalidDisputeStatementKind),
+}
+
+```
+
+## Dispute Statement Kinds
+
+Kinds of dispute statements. Each of these can be combined with a candidate hash, session index, validator public key, and validator signature to reproduce and check the original statement.
+
+```rust
+enum ValidDisputeStatementKind {
+    Explicit,
+    BackingSeconded,
+    BackingValid,
+    ApprovalChecking,
+}
+
+enum InvalidDisputeStatementKind {
+    Explicit,
+}
+```
+
+## ExplicitDisputeStatement
+
+```rust
+struct ExplicitDisputeStatement {
+    valid: bool,
+    candidate_hash: CandidateHash,
+    session: SessionIndex,
+}
+```
+
+## MultiDisputeStatementSet
+
+Sets of statements for many (zero or more) disputes.
+
+```rust
+type MultiDisputeStatementSet = Vec<DisputeStatementSet>;
+```
+
+## DisputeState
+
+```rust
+struct DisputeState {
+    validators_for: Bitfield, // one bit per validator.
+    validators_against: Bitfield, // one bit per validator.
+    start: BlockNumber,
+    concluded_at: Option<BlockNumber>,
+}
+```

roadmap/implementers-guide/src/types/overseer-protocol.md

@@ -449,11 +449,6 @@ enum ProvisionableData {
   Dispute(Hash, Signature),
 }
 
-/// This data needs to make its way from the provisioner into the InherentData.
-///
-/// There, it is used to construct the InclusionInherent.
-type ProvisionerInherentData = (SignedAvailabilityBitfields, Vec<BackedCandidate>);
-
 /// Message to the Provisioner.
 ///
 /// In all cases, the Hash is that of the relay parent.
@@ -461,12 +456,14 @@ enum ProvisionerMessage {
   /// This message allows potential block authors to be kept updated with all new authorship data
   /// as it becomes available.
   RequestBlockAuthorshipData(Hash, Sender<ProvisionableData>),
-  /// This message allows external subsystems to request the set of bitfields and backed candidates
-  /// associated with a particular potential block hash.
+  /// This message allows external subsystems to request current inherent data that could be used for
+  /// advancing the state of parachain consensus in a block building upon the given hash.
+  ///
+  /// If called at different points in time, this may give different results.
   ///
   /// This is expected to be used by a proposer, to inject that information into the InherentData
-  /// where it can be assembled into the InclusionInherent.
-  RequestInherentData(Hash, oneshot::Sender<ProvisionerInherentData>),
+  /// where it can be assembled into the ParaInherent.
+  RequestInherentData(Hash, oneshot::Sender<ParaInherentData>),
   /// This data should become part of a relay chain block
   ProvisionableData(ProvisionableData),
 }

roadmap/implementers-guide/src/types/runtime.md

@@ -105,3 +105,20 @@ struct HostConfiguration {
 	pub hrmp_max_message_num_per_candidate: u32,
 }
 ```
+
+## ParaInherentData
+
+Inherent data passed to a runtime entry-point for the advancement of parachain consensus.
+
+This contains 3 pieces of data:
+1. [`Bitfields`](availability.md#signed-availability-bitfield) 
+2. [`BackedCandidates`](backing.md#backed-candidate)
+3. [`MultiDisputeStatementSet`](disputes.md#multidisputestatementset)
+
+```rust
+struct ParaInherentData {
+	bitfields: Bitfields,
+	backed_candidates: BackedCandidates,
+	dispute_statements: MultiDisputeStatementSet,
+}
+```