Unverified Commit 9203072f authored by asynchronous rob's avatar asynchronous rob Committed by GitHub
Browse files

Various Guide Improvements (#1270)



* types chapter & candidate types

* Factor out some message types

* backing and availability types.

* spacing

* finish thought

* info on signed payloads

* update links

* explicit wrapper types

* add remaining message types

* add runtime API subsystem

* adjust language

* tweak candidate validation message

* find homes for some misfit types

* embed abridged within full candidate receipt

* Update roadmap/implementors-guide/src/SUMMARY.md
Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Update roadmap/implementors-guide/src/SUMMARY.md
Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* adjust text on candidate selection message to match
Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
parent cb42ab85
Pipeline #97262 passed with stages
in 19 minutes and 51 seconds
......@@ -36,7 +36,14 @@
- [Network Bridge](node/utility/network-bridge.md)
- [Misbehavior Arbitration](node/utility/misbehavior-arbitration.md)
- [Peer Set Manager](node/utility/peer-set-manager.md)
- [Data Structures and Types](type-definitions.md)
- [Runtime API Requests](node/utility/runtime-api.md)
- [Data Structures and Types](types/README.md)
- [Candidate](types/candidate.md)
- [Backing](types/backing.md)
- [Availability](types/availability.md)
- [Overseer and Subsystem Protocol](types/overseer-protocol.md)
- [Runtime](types/runtime.md)
- [Chain](types/chain.md)
[Glossary](glossary.md)
[Further Reading](further-reading.md)
......@@ -6,11 +6,7 @@ Validators vote on the availability of a backed candidate by issuing signed bitf
`ProtocolId`: `b"bitd"`
Input:
- `DistributeBitfield(relay_parent, SignedAvailabilityBitfield)`: distribute a bitfield via gossip to other validators.
- `NetworkBridgeUpdate(NetworkBridgeUpdate)`
Input: [`BitfieldDistributionMessage`](../../overseer-protocol.md#bitfield-distribution-message)
Output:
- `NetworkBridge::RegisterEventProducer(ProtocolId)`
......
......@@ -2,7 +2,7 @@
The Candidate Backing subsystem ensures every parablock considered for relay block inclusion has been seconded by at least one validator, and approved by a quorum. Parablocks for which no validator will assert correctness are discarded. If the block later proves invalid, the initial backers are slashable; this gives polkadot a rational threat model during subsequent stages.
Its role is to produce backable candidates for inclusion in new relay-chain blocks. It does so by issuing signed [`Statement`s](../../type-definitions.html#statement-type) and tracking received statements signed by other validators. Once enough statements are received, they can be combined into backing for specific candidates.
Its role is to produce backable candidates for inclusion in new relay-chain blocks. It does so by issuing signed [`Statement`s](../../types/backing.html#statement-type) and tracking received statements signed by other validators. Once enough statements are received, they can be combined into backing for specific candidates.
Note that though the candidate backing subsystem attempts to produce as many backable candidates as possible, it does _not_ attempt to choose a single authoritative one. The choice of which actually gets included is ultimately up to the block author, by whatever metrics it may use; those are opaque to this subsystem.
......@@ -10,9 +10,9 @@ Once a sufficient quorum has agreed that a candidate is valid, this subsystem no
## Protocol
The [Candidate Selection subsystem](candidate-selection.html) is the primary source of non-overseer messages into this subsystem. That subsystem generates appropriate [`CandidateBackingMessage`s](../../type-definitions.html#candidate-backing-message), and passes them to this subsystem.
The [Candidate Selection subsystem](candidate-selection.html) is the primary source of non-overseer messages into this subsystem. That subsystem generates appropriate [`CandidateBackingMessage`s](../../types/overseer-protocol.html#candidate-backing-message), and passes them to this subsystem.
This subsystem validates the candidates and generates an appropriate [`Statement`](../../type-definitions.html#statement-type). All `Statement`s are then passed on to the [Statement Distribution subsystem](statement-distribution.html) to be gossiped to peers. When this subsystem decides that a candidate is invalid, and it was recommended to us to second by our own Candidate Selection subsystem, a message is sent to the Candidate Selection subsystem with the candidate's hash so that the collator which recommended it can be penalized.
This subsystem validates the candidates and generates an appropriate [`Statement`](../../types/backing.html#statement-type). All `Statement`s are then passed on to the [Statement Distribution subsystem](statement-distribution.html) to be gossiped to peers. When this subsystem decides that a candidate is invalid, and it was recommended to us to second by our own Candidate Selection subsystem, a message is sent to the Candidate Selection subsystem with the candidate's hash so that the collator which recommended it can be penalized.
## Functionality
......@@ -20,8 +20,8 @@ The subsystem should maintain a set of handles to Candidate Backing Jobs that ar
### On Overseer Signal
* If the signal is an [`OverseerSignal`](../../type-definitions.html#overseer-signal)`::StartWork(relay_parent)`, spawn a Candidate Backing Job with the given relay parent, storing a bidirectional channel with the Candidate Backing Job in the set of handles.
* If the signal is an [`OverseerSignal`](../../type-definitions.html#overseer-signal)`::StopWork(relay_parent)`, cease the Candidate Backing Job under that relay parent, if any.
* If the signal is an [`OverseerSignal`](../../types/overseer-protocol.html#overseer-signal)`::StartWork(relay_parent)`, spawn a Candidate Backing Job with the given relay parent, storing a bidirectional channel with the Candidate Backing Job in the set of handles.
* If the signal is an [`OverseerSignal`](../../types/overseer-protocol.html#overseer-signal)`::StopWork(relay_parent)`, cease the Candidate Backing Job under that relay parent, if any.
### On `CandidateBackingMessage`
......@@ -39,7 +39,7 @@ The subsystem should maintain a set of handles to Candidate Backing Jobs that ar
The Candidate Backing Job represents the work a node does for backing candidates with respect to a particular relay-parent.
The goal of a Candidate Backing Job is to produce as many backable candidates as possible. This is done via signed [`Statement`s](../../type-definitions.html#statement-type) by validators. If a candidate receives a majority of supporting Statements from the Parachain Validators currently assigned, then that candidate is considered backable.
The goal of a Candidate Backing Job is to produce as many backable candidates as possible. This is done via signed [`Statement`s](../../types/backing.html#statement-type) by validators. If a candidate receives a majority of supporting Statements from the Parachain Validators currently assigned, then that candidate is considered backable.
### On Startup
......
......@@ -6,18 +6,18 @@ This subsystem includes networking code for communicating with collators, and tr
This subsystem is only ever interested in parablocks assigned to the particular parachain which this validator is currently handling.
New parablock candidates may arrive from a potentially unbounded set of collators. This subsystem chooses either 0 or 1 of them per relay parent to second. If it chooses to second a candidate, it sends an appropriate message to the [Candidate Backing subsystem](candidate-backing.html) to generate an appropriate [`Statement`](../../type-definitions.html#statement-type).
New parablock candidates may arrive from a potentially unbounded set of collators. This subsystem chooses either 0 or 1 of them per relay parent to second. If it chooses to second a candidate, it sends an appropriate message to the [Candidate Backing subsystem](candidate-backing.html) to generate an appropriate [`Statement`](../../types/backing.html#statement-type).
In the event that a parablock candidate proves invalid, this subsystem will receive a message back from the Candidate Backing subsystem indicating so. If that parablock candidate originated from a collator, this subsystem will blacklist that collator. If that parablock candidate originated from a peer, this subsystem generates a report for the [Misbehavior Arbitration subsystem](../utility/misbehavior-arbitration.html).
## Protocol
Input: None from other subsystems. This subsystem contains internal network logic for receiving parablock candidates and associated Proofs of Validity from collators.
Input: [`CandidateSelectionMessage`](../../types/overseer-protocol#candidate-selection-message)
Output:
- Validation requests to Validation subsystem
- [`CandidateBackingMessage`](../../type-definitions.html#candidate-backing-message)`::Second`
- [`CandidateBackingMessage`](../../types/overseer-protocol.html#candidate-backing-message)`::Second`
- Peer set manager: report peers (collators who have misbehaved)
## Functionality
......
......@@ -35,7 +35,7 @@ The Statement Distribution subsystem sends statements to peer nodes and detects
## Peer Receipt State Machine
There is a very simple state machine which governs which messages we are willing to receive from peers. Not depicted in the state machine: on initial receipt of any [`SignedStatement`](../../type-definitions.html#signed-statement-type), validate that the provided signature does in fact sign the included data. Note that each individual parablock candidate gets its own instance of this state machine; it is perfectly legal to receive a `Valid(X)` before a `Seconded(Y)`, as long as a `Seconded(X)` has been received.
There is a very simple state machine which governs which messages we are willing to receive from peers. Not depicted in the state machine: on initial receipt of any [`SignedStatement`](../../types/backing.html#signed-statement-type), validate that the provided signature does in fact sign the included data. Note that each individual parablock candidate gets its own instance of this state machine; it is perfectly legal to receive a `Valid(X)` before a `Seconded(Y)`, as long as a `Seconded(X)` has been received.
A: Initial State. Receive `SignedStatement(Statement::Second)`: extract `Statement`, forward to Candidate Backing, proceed to B. Receive any other `SignedStatement` variant: drop it.
B: Receive any `SignedStatement`: extract `Statement`, forward to Candidate Backing. Receive `OverseerMessage::StopWork`: proceed to C.
......
......@@ -24,7 +24,7 @@ The hierarchy of subsystems:
```
The overseer determines work to do based on block import events and block finalization events. It does this by keeping track of the set of relay-parents for which work is currently being done. This is known as the "active leaves" set. It determines an initial set of active leaves on startup based on the data on-disk, and uses events about blockchain import to update the active leaves. Updates lead to [`OverseerSignal`](../type-definitions.html#overseer-signal)`::StartWork` and [`OverseerSignal`](../type-definitions.html#overseer-signal)`::StopWork` being sent according to new relay-parents, as well as relay-parents to stop considering. Block import events inform the overseer of leaves that no longer need to be built on, now that they have children, and inform us to begin building on those children. Block finalization events inform us when we can stop focusing on blocks that appear to have been orphaned.
The overseer determines work to do based on block import events and block finalization events. It does this by keeping track of the set of relay-parents for which work is currently being done. This is known as the "active leaves" set. It determines an initial set of active leaves on startup based on the data on-disk, and uses events about blockchain import to update the active leaves. Updates lead to [`OverseerSignal`](../types.overseer-protocol.html#overseer-signal)`::StartWork` and [`OverseerSignal`](../types/overseer-protocol.html#overseer-signal)`::StopWork` being sent according to new relay-parents, as well as relay-parents to stop considering. Block import events inform the overseer of leaves that no longer need to be built on, now that they have children, and inform us to begin building on those children. Block finalization events inform us when we can stop focusing on blocks that appear to have been orphaned.
The overseer's logic can be described with these functions:
......
......@@ -25,11 +25,7 @@ There may be multiple competing blocks all ending the availability phase for a p
## Protocol
Input:
- QueryPoV(candidate_hash, response_channel)
- QueryChunk(candidate_hash, validator_index, response_channel)
- StoreChunk(candidate_hash, validator_index, inclusion_proof, chunk_data)
Input: [`AvailabilityStoreMessage`](../../types/overseer-protocol.html#availability-store-message)
## Functionality
......@@ -49,7 +45,7 @@ On `QueryPoV` message:
On `QueryChunk` message:
- Determine if we have the chunk indicated by the parameters and return it via the response channel if so.
- Determine if we have the chunk indicated by the parameters and return it and its inclusion proof via the response channel if so.
On `StoreChunk` message:
......
......@@ -6,9 +6,9 @@ A variety of subsystems want to know if a parachain block candidate is valid. No
## Protocol
Input: [`CandidateValidationMessage`](../../type-definitions.html#validation-request-type)
Input: [`CandidateValidationMessage`](../../types/overseer-protocol.html#validation-request-type)
Output: [`Statement`](../../type-definitions.html#statement-type) via the provided `Sender<Statement>`.
Output: Validation result via the provided response side-channel.
## Functionality
......
......@@ -10,27 +10,8 @@ So in short, this Subsystem acts as a bridge between an actual network component
## Protocol
> REVIEW: I am designing this using dynamic dispatch based on a ProtocolId discriminant rather than doing static dispatch to specific subsystems based on a concrete network message type. The reason for this is that doing static dispatch might break the property that Subsystem implementations can be swapped out for others. So this is actually implementing a subprotocol multiplexer. Pierre tells me this is OK for our use-case ;). One caveat is that now all network traffic will also flow through the overseer, but this overhead is probably OK.
```rust
use sc-network::ObservedRole;
struct View(Vec<Hash>); // Up to `N` (5?) chain heads.
enum NetworkBridgeEvent {
PeerConnected(PeerId, ObservedRole), // role is one of Full, Light, OurGuardedAuthority, OurSentry
PeerDisconnected(PeerId),
PeerMessage(PeerId, Bytes),
PeerViewChange(PeerId, View), // guaranteed to come after peer connected event.
OurViewChange(View),
}
```
Input:
- RegisterEventProducer(`ProtocolId`, `Fn(NetworkBridgeEvent) -> AllMessages`): call on startup.
- ReportPeer(PeerId, cost_or_benefit)
- SendMessage(`[PeerId]`, `ProtocolId`, Bytes): send a message to multiple peers.
Input: [`NetworkBridgeMessage`](../../types/overseer-protocol.html#network-bridge-message)
Output: Varying, based on registered event producers.
## Functionality
......
......@@ -10,11 +10,11 @@ There are several distinct types of provisionable data, but they share this prop
### Backed Candidates
The block author can choose 0 or 1 backed parachain candidates per parachain; the only constraint is that each backed candidate has the appropriate relay parent. However, the choice of a backed candidate must be the block author's; the provisioner must ensure that block authors are aware of all available [`BackedCandidate`s](../../type-definitions.html#backed-candidate).
The block author can choose 0 or 1 backed parachain candidates per parachain; the only constraint is that each backed candidate has the appropriate relay parent. However, the choice of a backed candidate must be the block author's; the provisioner must ensure that block authors are aware of all available [`BackedCandidate`s](../../types/backing.html#backed-candidate).
### Signed Bitfields
[Signed bitfields](../../type-definitions.html#signed-availability-bitfield) are attestations from a particular validator about which candidates it believes are available.
[Signed bitfields](../../types/availability.html#signed-availability-bitfield) are attestations from a particular validator about which candidates it believes are available.
### Misbehavior Reports
......@@ -32,7 +32,7 @@ Dispute resolution is complex and is explained in substantially more detail [her
## Protocol
Input: [`ProvisionerMessage`](../../type-definitions.html#provisioner-message). Backed candidates come from the [Candidate Backing subsystem](../backing/candidate-backing.html), signed bitfields come from the [Bitfield Distribution subsystem](../availability/bitfield-distribution.html), and misbehavior reports and disputes come from the [Misbehavior Arbitration subsystem](misbehavior-arbitration.html).
Input: [`ProvisionerMessage`](../../types/overseer-protocol.html#provisioner-message). Backed candidates come from the [Candidate Backing subsystem](../backing/candidate-backing.html), signed bitfields come from the [Bitfield Distribution subsystem](../availability/bitfield-distribution.html), and misbehavior reports and disputes come from the [Misbehavior Arbitration subsystem](misbehavior-arbitration.html).
At initialization, this subsystem has no outputs. Block authors can send a `ProvisionerMessage::RequestBlockAuthorshipData`, which includes a channel over which provisionable data can be sent. All appropriate provisionable data will then be sent over this channel, as it is received.
......
# Runtime API
The Runtime API subsystem is responsible for providing a single point of access to runtime state data via a set of pre-determined queries. This prevents shared ownership of a blockchain client resource by providing
## Protocol
Input: [`RuntimeApiMessage`](../../types/overseer-protocol.html#runtime-api-message)
Output: None
## Functionality
On receipt of `RuntimeApiMessage::Request(relay_parent, request)`, answer the request using the post-state of the relay_parent provided and provide the response to the side-channel embedded within the request.
> TODO Do some caching. The underlying rocksdb already has a cache of trie nodes so duplicate requests are unlikely to hit disk. Not required for functionality.
## Jobs
> TODO Don't limit requests based on parent hash, but limit caching. No caching should be done for any requests on relay_parents that are not live based on `StartWork` or `StopWork` messages. Maybe with some leeway for things that have just been stopped.
......@@ -2,7 +2,7 @@
This module is responsible for managing all configuration of the parachain host in-flight. It provides a central point for configuration updates to prevent races between configuration changes and parachain-processing logic. Configuration can only change during the session change routine, and as this module handles the session change notification first it provides an invariant that the configuration does not change throughout the entire session. Both the [scheduler](scheduler.html) and [inclusion](inclusion.html) modules rely on this invariant to ensure proper behavior of the scheduler.
The configuration that we will be tracking is the [`HostConfiguration`](../type-definitions.html#host-configuration) struct.
The configuration that we will be tracking is the [`HostConfiguration`](../types/runtime.html#host-configuration) struct.
## Storage
......
......@@ -16,7 +16,7 @@ Included: Option<()>,
## Entry Points
* `inclusion`: This entry-point accepts two parameters: [`Bitfields`](../type-definitions.html#signed-availability-bitfield) and [`BackedCandidates`](../type-definitions.html#backed-candidate).
* `inclusion`: This entry-point accepts two parameters: [`Bitfields`](../types/availability.html#signed-availability-bitfield) and [`BackedCandidates`](../types/backing.html#backed-candidate).
1. The `Bitfields` are first forwarded to the `process_bitfields` routine, returning a set of freed cores. Provide a `Scheduler::core_para` as a core-lookup to the `process_bitfields` routine. Annotate each of these freed cores with `FreedReason::Concluded`.
1. If `Scheduler::availability_timeout_predicate` is `Some`, invoke `Inclusion::collect_pending` using it, and add timed-out cores to the free cores, annotated with `FreedReason::TimedOut`.
1. Invoke `Scheduler::schedule(freed)`
......
......@@ -64,7 +64,7 @@ Validator group assignments do not need to change very quickly. The security ben
Validator groups rotate across availability cores in a round-robin fashion, with rotation occurring at fixed intervals. The i'th group will be assigned to the `(i+k)%n`'th core at any point in time, where `k` is the number of rotations that have occurred in the session, and `n` is the number of cores. This makes upcoming rotations within the same session predictable.
When a rotation occurs, validator groups are still responsible for distributing availability chunks for any previous cores that are still occupied and pending availability. In practice, rotation and availability-timeout frequencies should be set so this will only be the core they have just been rotated from. It is possible that a validator group is rotated onto a core which is currently occupied. In this case, the validator group will have nothing to do until the previously-assigned group finishes their availability work and frees the core or the availability process times out. Depending on if the core is for a parachain or parathread, a different timeout `t` from the [`HostConfiguration`](../type-definitions.html#host-configuration) will apply. Availability timeouts should only be triggered in the first `t-1` blocks after the beginning of a rotation.
When a rotation occurs, validator groups are still responsible for distributing availability chunks for any previous cores that are still occupied and pending availability. In practice, rotation and availability-timeout frequencies should be set so this will only be the core they have just been rotated from. It is possible that a validator group is rotated onto a core which is currently occupied. In this case, the validator group will have nothing to do until the previously-assigned group finishes their availability work and frees the core or the availability process times out. Depending on if the core is for a parachain or parathread, a different timeout `t` from the [`HostConfiguration`](../types/runtime.html#host-configuration) will apply. Availability timeouts should only be triggered in the first `t-1` blocks after the beginning of a rotation.
Parathreads operate on a system of claims. Collators participate in auctions to stake a claim on authoring the next block of a parathread, although the auction mechanism is beyond the scope of the scheduler. The scheduler guarantees that they'll be given at least a certain number of attempts to author a candidate that is backed. Attempts that fail during the availability phase are not counted, since ensuring availability at that stage is the responsibility of the backing validators, not of the collator. When a claim is accepted, it is placed into a queue of claims, and each claim is assigned to a particular parathread-multiplexing core in advance. Given that the current assignments of validator groups to cores are known, and the upcoming assignments are predictable, it is possible for parathread collators to know who they should be talking to now and how they should begin establishing connections with as a fallback.
......@@ -153,7 +153,7 @@ Actions:
1. Set `SessionStartBlock` to current block number.
1. Clear all `Some` members of `AvailabilityCores`. Return all parathread claims to queue with retries un-incremented.
1. Set `configuration = Configuration::configuration()` (see [`HostConfiguration`](../type-definitions.html#host-configuration))
1. Set `configuration = Configuration::configuration()` (see [`HostConfiguration`](../types/runtime.html#host-configuration))
1. Resize `AvailabilityCores` to have length `Paras::parachains().len() + configuration.parathread_cores with all`None` entries.
1. Compute new validator groups by shuffling using a secure randomness beacon
- We need a total of `N = Paras::parathreads().len() + configuration.parathread_cores` validator groups.
......
# Type Definitions
This section of the guide provides type definitions of various categories.
# Availability
One of the key roles of validators is to ensure availability of all data necessary to validate
candidates for the duration of a challenge period. This is done via an erasure-coding of the data to keep available.
## Signed Availability Bitfield
A bitfield signed by a particular validator about the availability of pending candidates.
```rust
struct SignedAvailabilityBitfield {
validator_index: ValidatorIndex,
bitfield: Bitvec,
signature: ValidatorSignature,
}
struct Bitfields(Vec<(SignedAvailabilityBitfield)>), // bitfields sorted by validator index, ascending
```
The signed payload is the SCALE encoding of the tuple `(bitfield, signing_context)` where `signing_context` is a [`SigningContext`](../types/candidate.html#signing-context).
## Proof-of-Validity
Often referred to as PoV, this is a type-safe wrapper around bytes (`Vec<u8>`) when referring to data that acts as a stateless-client proof of validity of a candidate, when used as input to the validation function of the para.
```rust
struct PoV(Vec<u8>);
```
# Backing Types
[Candidates](candidate.html) go through many phases before being considered included in a fork of the relay chain and eventually accepted.
These types describe the data used in the backing phase. Some are sent over the wire within subsystems, and some are simply included in the relay-chain block.
## Validity Attestation
An attestation of validity for a candidate, used as part of a backing. Both the `Seconded` and `Valid` statements are considered attestations of validity. This structure is only useful where the candidate referenced is apparent.
```rust
enum ValidityAttestation {
/// Implicit validity attestation by issuing.
/// This corresponds to issuance of a `Seconded` statement.
Implicit(ValidatorSignature),
/// An explicit attestation. This corresponds to issuance of a
/// `Valid` statement.
Explicit(ValidatorSignature),
}
```
## Statement Type
The [Candidate Backing subsystem](../node/backing/candidate-backing.html) issues and signs these after candidate validation.
```rust
/// A statement about the validity of a parachain candidate.
enum Statement {
/// A statement about a new candidate being seconded by a validator. This is an implicit validity vote.
///
/// The main semantic difference between `Seconded` and `Valid` comes from the fact that every validator may
/// second only 1 candidate; this places an upper bound on the total number of candidates whose validity
/// needs to be checked. A validator who seconds more than 1 parachain candidate per relay head is subject
/// to slashing.
Seconded(CandidateReceipt),
/// A statement about the validity of a candidate, based on candidate's hash.
Valid(Hash),
/// A statement about the invalidity of a candidate.
Invalid(Hash),
}
```
## Signed Statement Type
A statement, the identifier of a validator, and a signature.
```rust
/// A signed statement.
struct SignedStatement {
/// The index of the validator signing this statement.
validator_index: ValidatorIndex,
/// The statement itself.
statement: Statement,
/// The signature by the validator on the signing payload.
signature: ValidatorSignature
}
```
The actual signed payload will be the SCALE encoding of `(compact_statement, signing_context)` where
`compact_statement` is a tweak of the [`Statement`](#statement) enum where all variants, including `Seconded`, contain only the hash of the candidate, and the `signing_context` is a [`SigningContext`](../types/candidate.html#signing-context).
This prevents against replay attacks and allows the candidate receipt itself to be omitted when checking a signature on a `Seconded` statement in situations where the hash is known.
## Backed Candidate
An [`AbridgedCandidateReceipt`](candidate.html#abridgedcandidatereceipt) along with all data necessary to prove its backing. This is submitted to the relay-chain to process and move along the candidate to the pending-availability stage.
```rust
struct BackedCandidate {
candidate: AbridgedCandidateReceipt,
validity_votes: Vec<ValidityAttestation>,
// the indices of validators who signed the candidate within the group. There is no need to include
// bit for any validators who are not in the group, so this is more compact.
// The number of bits is the number of validators in the group.
//
// the group should be apparent from context.
validator_indices: BitVec,
}
struct BackedCandidates(Vec<BackedCandidate>); // sorted by para-id.
```
# Candidate Types
Para candidates are some of the most common types, both within the runtime and on the Node-side.
In a way, this entire guide is about these candidates: how they are scheduled, constructed, backed, included, and challenged.
This section will describe the base candidate type, its components, and abridged counterpart.
## CandidateReceipt
This is the base receipt type. The `GlobalValidationSchedule` and the `LocalValidationData` are technically redundant with the `inner.relay_parent`, which uniquely describes the a block in the blockchain from whose state these values are derived. The [`AbridgedCandidateReceipt`](#abridgedcandidatereceipt) variant is often used instead for this reason.
However, the full CandidateReceipt type is useful as a means of avoiding the implicit dependency on availability of old blockchain state. In situations such as availability and approval, having the full description of the candidate within a self-contained struct is convenient.
```rust
/// All data pertaining to the execution of a para candidate.
struct CandidateReceipt {
inner: AbridgedCandidateReceipt,
/// The global validation schedule.
global_validation: GlobalValidationSchedule,
/// The local validation data.
local_validation: LocalValidationData,
}
```
## AbridgedCandidateReceipt
Much info in a [`CandidateReceipt`](#candidatereceipt) is duplicated from the relay-chain state. When the corresponding relay-chain state is considered widely available, the Abridged Candidate Receipt should be favored.
Examples of situations where the state is readily available includes within the scope of work done by subsystems working on a given relay-parent, or within the logic of the runtime importing a backed candidate.
```rust
/// An abridged candidate-receipt.
struct AbridgedCandidateReceipt {
/// The ID of the para this is a candidate for.
para_id: Id,
/// The hash of the relay-chain block this is executed in the context of.
relay_parent: Hash,
/// The head-data produced as a result of execution.
head_data: HeadData,
/// The collator's sr25519 public key.
collator: CollatorId,
/// Signature on blake2-256 of components of this receipt:
/// The parachain index, the relay parent, the head data, and the pov_hash.
signature: CollatorSignature,
/// The blake2-256 hash of the pov-block.
pov_hash: Hash,
/// Commitments made as a result of validation.
commitments: CandidateCommitments,
}
```
## GlobalValidationSchedule
The global validation schedule comprises of information describing the global environment for para execution, as derived from a particular relay-parent. These are parameters that will apply to all parablocks executed in the context of this relay-parent.
> TODO: message queue watermarks (first upward messages, then XCMP channels)
```rust
/// Extra data that is needed along with the other fields in a `CandidateReceipt`
/// to fully validate the candidate.
///
/// These are global parameters that apply to all candidates in a block.
struct GlobalValidationSchedule {
/// The maximum code size permitted, in bytes.
max_code_size: u32,
/// The maximum head-data size permitted, in bytes.
max_head_data_size: u32,
/// The relay-chain block number this is in the context of.
block_number: BlockNumber,
}
```
## LocalValidationData
This is validation data needed for execution of candidate pertaining to a specific para and relay-chain block.
Unlike the [`GlobalValidationData`](#globalvalidationdata), which only depends on a relay-parent, this is parameterized both by a relay-parent and a choice of one of two options:
1. Assume that the candidate pending availability on this para at the onset of the relay-parent is included.
1. Assume that the candidate pending availability on this para at the onset of the relay-parent is timed-out.
This choice can also be expressed as a choice of which parent head of the para will be built on - either optimistically on the candidate pending availability or pessimistically on the one that is surely included.
Para validation happens optimistically before the block is authored, so it is not possible to predict with 100% accuracy what will happen in the earlier phase of the [`InclusionInherent`](/runtime/inclusioninherent.html) module where new availability bitfields and availability timeouts are processed. This is what will eventually define whether a candidate can be backed within a specific relay-chain block.
> TODO: determine if balance/fees are even needed here.
```rust
/// Extra data that is needed along with the other fields in a `CandidateReceipt`
/// to fully validate the candidate. These fields are parachain-specific.
pub struct LocalValidationData {
/// The parent head-data.
pub parent_head: HeadData,
/// The balance of the parachain at the moment of validation.
pub balance: Balance,
/// The blake2-256 hash of the validation code used to execute the candidate.
pub validation_code_hash: Hash,
/// Whether the parachain is allowed to upgrade its validation code.
///
/// This is `Some` if so, and contains the number of the minimum relay-chain
/// height at which the upgrade will be applied, if an upgrade is signaled
/// now.
///
/// A parachain should enact its side of the upgrade at the end of the first
/// parablock executing in the context of a relay-chain block with at least this
/// height. This may be equal to the current perceived relay-chain block height, in
/// which case the code upgrade should be applied at the end of the signaling
/// block.
pub code_upgrade_allowed: Option<BlockNumber>,
}
```
## HeadData
Head data is a type-safe abstraction around bytes (`Vec<u8>`) for the purposes of representing heads of parachains or parathreads.
```rust
struct HeadData(Vec<u8>);
```
## CandidateCommitments
The execution and validation of parachain or parathread candidates produces a number of values which either must be committed to on the relay chain or committed to the state of the relay chain.
```rust
/// Commitments made in a `CandidateReceipt`. Many of these are outputs of validation.
#[derive(PartialEq, Eq, Clone, Encode, Decode)]
#[cfg_attr(feature = "std", derive(Debug, Default))]
pub struct CandidateCommitments {
/// Fees paid from the chain to the relay chain validators.
pub fees: Balance,
/// Messages destined to be interpreted by the Relay chain itself.
pub upward_messages: Vec<UpwardMessage>,
/// The root of a block's erasure encoding Merkle tree.
pub erasure_root: Hash,
/// New validation code.
pub new_validation_code: Option<ValidationCode>,
}
```
## Signing Context
This struct provides context to signatures by combining with various payloads to localize the signature to a particular session index and relay-chain hash. Having these fields included in the signature makes misbehavior attribution much simpler.
```rust
struct SigningContext {
/// The relay-chain block hash this signature is in the context of.
parent_hash: Hash,
/// The session index this signature is in the context of.
session_index: SessionIndex,
}
```
# Chain
Types pertaining to the relay-chain - events, structures, etc.
## Block Import Event
```rust
/// Indicates that a new block has been added to the blockchain.
struct BlockImportEvent {
/// The block header-hash.
hash: Hash,
/// The header itself.
header: Header,
/// Whether this block is considered the head of the best chain according to the
/// event emitter's fork-choice rule.
new_best: bool,
}
```
## Block Finalization Event
```rust
/// Indicates that a new block has been finalized.
struct BlockFinalizationEvent {
/// The block header-hash.
hash: Hash,
/// The header of the finalized block.
header: Header,
}
```
# Data Structures and Types
# Overseer Protocol
> TODO:
>
> * CandidateReceipt
> * CandidateCommitments
> * AbridgedCandidateReceipt
> * GlobalValidationSchedule
> * LocalValidationData (should commit to code hash too - see Remote disputes section of validity module)
## Block Import Event
```rust
/// Indicates that a new block has been added to the blockchain.
struct BlockImportEvent {
/// The block header-hash.
hash: Hash,
/// The header itself.
header: Header,
/// Whether this block is considered the head of the best chain according to the
/// event emitter's fork-choice rule.
new_best: bool,
}
```