Unverified Commit 217083a9 authored by asynchronous rob's avatar asynchronous rob Committed by GitHub
Browse files

New parachain runtime skeleton (#1158)

* file structure and initializer skeleton

* ensure session changes happen before initialization

* add a couple tests for initializer flow

* integrate with session handling

* configuration update logic

* configuration methods

* move test mock to its own module

* integrate configuration into initializer

* add note about initialization order

* integrate configuration module into mock

* add some tests for config module

* paras module storage

* implement paras session change operation

* amend past code pruning to fully cover acceptance period

* update guide again

* do pruning of historical validation code

* add weight to initialization

* integrate into mock & leave notes for next session

* clean up un-ended sentence

* alter test to account for double index in past code meta

* port over code-at logic test

* clarify checking for conflicting code upgrades

* add genesis for paras, include in mock, ensure incoming paras are processed

* note on return value of `validation_code_at`

* implement paras routines from implementor's guide

* bring over some existing tests and begin porting

* port over code upgrade tests

* test parachain registration

* test code_at with intermediate block

* fix warnings

* clean up docs and extract to separate struct

* adjust implementor's guide to include replacementtimes

* kill stray println

* rename expected_at to applied_after

* rewrite ParaPastCodeMeta to avoid reversal

* clarify and test interface of validation_code_at

* make FutureCode optional

* rename do_old_code_pruning

* add comment on Option<()> to answer FAQ

* address some more grumbles
parent cc7f6bfe
Pipeline #95027 passed with stages
in 20 minutes and 24 seconds
......@@ -4457,6 +4457,48 @@ dependencies = [
"trie-db",
]
[[package]]
name = "polkadot-runtime-parachains"
version = "0.8.0"
dependencies = [
"bitvec",
"frame-benchmarking",
"frame-support",
"frame-system",
"hex-literal",
"libsecp256k1",
"log 0.3.9",
"pallet-authorship",
"pallet-babe",
"pallet-balances",
"pallet-offences",
"pallet-randomness-collective-flip",
"pallet-session",
"pallet-staking",
"pallet-staking-reward-curve",
"pallet-timestamp",
"pallet-treasury",
"pallet-vesting",
"parity-scale-codec",
"polkadot-primitives",
"rustc-hex",
"serde",
"serde_derive",
"serde_json",
"sp-api",
"sp-application-crypto",
"sp-core",
"sp-inherents",
"sp-io",
"sp-keyring",
"sp-runtime",
"sp-session",
"sp-staking",
"sp-std",
"sp-trie",
"trie-db",
]
[[package]]
name = "polkadot-service"
version = "0.8.2"
......
......@@ -30,6 +30,7 @@ members = [
"overseer",
"primitives",
"runtime/common",
"runtime/parachains",
"runtime/polkadot",
"runtime/kusama",
"runtime/westend",
......
......@@ -414,7 +414,7 @@ It's also responsible for managing parachain validation code upgrades as well as
Utility structs:
```rust
// the two key times necessary to track for every code replacement.
struct ReplacementTimes {
pub struct ReplacementTimes {
/// The relay-chain block number that the code upgrade was expected to be activated.
/// This is when the code change occurs from the para's perspective - after the
/// first parablock included with a relay-parent with number >= this value.
......@@ -481,7 +481,7 @@ PastCodePruning: Vec<(ParaId, BlockNumber)>;
/// in the context of a relay chain block with a number >= `expected_at`.
FutureCodeUpgrades: map ParaId => Option<BlockNumber>;
/// The actual future code of a para.
FutureCode: map ParaId => ValidationCode;
FutureCode: map ParaId => Option<ValidationCode>;
/// Upcoming paras (chains and threads). These are only updated on session change. Corresponds to an
/// entry in the upcoming-genesis map.
......@@ -507,7 +507,7 @@ OutgoingParas: Vec<ParaId>;
* `schedule_para_cleanup(ParaId)`: schedule a para to be cleaned up at the next session.
* `schedule_code_upgrade(ParaId, ValidationCode, expected_at: BlockNumber)`: Schedule a future code upgrade of the given parachain, to be applied after inclusion of a block of the same parachain executed in the context of a relay-chain block with number >= `expected_at`.
* `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head, where the new head was executed in the context of a relay-chain block with given number. This will apply pending code upgrades based on the block number provided.
* `validation_code_at(ParaId, at: BlockNumber, assume_intermediate: Option<BlockNumber>)`: Fetches the validation code to be used when validating a block in the context of the given relay-chain height. A second block number parameter may be used to tell the lookup to proceed as if an intermediate parablock has been included at the given relay-chain height. This may return past, current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if provided, must be before `at`. If `at` is not within `config.acceptance_period` of the current block number, this will return `None`.
* `validation_code_at(ParaId, at: BlockNumber, assume_intermediate: Option<BlockNumber>)`: Fetches the validation code to be used when validating a block in the context of the given relay-chain height. A second block number parameter may be used to tell the lookup to proceed as if an intermediate parablock has been included at the given relay-chain height. This may return past, current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if provided, must be before `at`. If the validation code has been pruned, this will return `None`.
#### Finalization
......@@ -755,6 +755,7 @@ All failed checks should lead to an unrecoverable error making the block invalid
1. Return a list of freed cores consisting of the cores where candidates have become available.
* `process_candidates(BackedCandidates, scheduled: Vec<CoreAssignment>)`:
1. check that each candidate corresponds to a scheduled core and that they are ordered in ascending order by `ParaId`.
1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_frequency` of the currently scheduled upgrade, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID.
1. check the backing of the candidate using the signatures and the bitfields.
1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes` bitfield.
1. Return a `Vec<CoreIndex>` of all scheduled cores of the list of passed assignments that a candidate was successfully backed for, sorted ascending by CoreIndex.
......
[package]
name = "polkadot-runtime-parachains"
version = "0.8.0"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2018"
[dependencies]
bitvec = { version = "0.17.4", default-features = false, features = ["alloc"] }
codec = { package = "parity-scale-codec", version = "1.3.0", default-features = false, features = ["derive"] }
log = { version = "0.3.9", optional = true }
rustc-hex = { version = "2.0.1", default-features = false }
serde = { version = "1.0.102", default-features = false }
serde_derive = { version = "1.0.102", optional = true }
sp-api = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
inherents = { package = "sp-inherents", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-std = { package = "sp-std", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-io = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-runtime = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-session = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-staking = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
authorship = { package = "pallet-authorship", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
balances = { package = "pallet-balances", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
session = { package = "pallet-session", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
frame-support = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
staking = { package = "pallet-staking", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
system = { package = "frame-system", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
timestamp = { package = "pallet-timestamp", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
vesting = { package = "pallet-vesting", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
offences = { package = "pallet-offences", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
frame-benchmarking = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false, optional = true }
primitives = { package = "polkadot-primitives", path = "../../primitives", default-features = false }
libsecp256k1 = { version = "0.3.2", default-features = false, optional = true }
[dev-dependencies]
hex-literal = "0.2.1"
keyring = { package = "sp-keyring", git = "https://github.com/paritytech/substrate", branch = "master" }
sp-trie = { git = "https://github.com/paritytech/substrate", branch = "master" }
babe = { package = "pallet-babe", git = "https://github.com/paritytech/substrate", branch = "master" }
sp-application-crypto = { git = "https://github.com/paritytech/substrate", branch = "master" }
randomness-collective-flip = { package = "pallet-randomness-collective-flip", git = "https://github.com/paritytech/substrate", branch = "master" }
pallet-staking-reward-curve = { git = "https://github.com/paritytech/substrate", branch = "master" }
treasury = { package = "pallet-treasury", git = "https://github.com/paritytech/substrate", branch = "master" }
trie-db = "0.20.0"
serde_json = "1.0.41"
libsecp256k1 = "0.3.2"
[features]
default = ["std"]
no_std = []
std = [
"bitvec/std",
"codec/std",
"log",
"rustc-hex/std",
"serde_derive",
"serde/std",
"primitives/std",
"inherents/std",
"sp-core/std",
"sp-api/std",
"sp-std/std",
"sp-io/std",
"frame-support/std",
"authorship/std",
"balances/std",
"sp-runtime/std",
"sp-session/std",
"sp-staking/std",
"session/std",
"staking/std",
"system/std",
"timestamp/std",
"vesting/std",
]
runtime-benchmarks = [
"libsecp256k1/hmac",
"frame-benchmarking",
"frame-support/runtime-benchmarks",
"system/runtime-benchmarks",
]
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Configuration manager for the Polkadot runtime parachains logic.
//!
//! Configuration can change only at session boundaries and is buffered until then.
use sp_std::prelude::*;
use primitives::{
parachain::{ValidatorId},
};
use frame_support::{
decl_storage, decl_module, decl_error,
dispatch::DispatchResult,
weights::{DispatchClass, Weight},
};
use codec::{Encode, Decode};
use system::ensure_root;
/// All configuration of the runtime with respect to parachains and parathreads.
#[derive(Clone, Encode, Decode, PartialEq, Default)]
#[cfg_attr(test, derive(Debug))]
pub struct HostConfiguration<BlockNumber: Default> {
/// The minimum frequency at which parachains can update their validation code.
pub validation_upgrade_frequency: BlockNumber,
/// The delay, in blocks, before a validation upgrade is applied.
pub validation_upgrade_delay: BlockNumber,
/// The acceptance period, in blocks. This is the amount of blocks after availability that validators
/// and fishermen have to perform secondary checks or issue reports.
pub acceptance_period: BlockNumber,
/// The maximum validation code size, in bytes.
pub max_code_size: u32,
/// The maximum head-data size, in bytes.
pub max_head_data_size: u32,
/// The amount of execution cores to dedicate to parathread execution.
pub parathread_cores: u32,
/// The number of retries that a parathread author has to submit their block.
pub parathread_retries: u32,
/// How often parachain groups should be rotated across parachains.
pub parachain_rotation_frequency: BlockNumber,
/// The availability period, in blocks, for parachains. This is the amount of blocks
/// after inclusion that validators have to make the block available and signal its availability to
/// the chain. Must be at least 1.
pub chain_availability_period: BlockNumber,
/// The availability period, in blocks, for parathreads. Same as the `chain_availability_period`,
/// but a differing timeout due to differing requirements. Must be at least 1.
pub thread_availability_period: BlockNumber,
/// The amount of blocks ahead to schedule parachains and parathreads.
pub scheduling_lookahead: u32,
}
pub trait Trait: system::Trait { }
decl_storage! {
trait Store for Module<T: Trait> as Configuration {
/// The active configuration for the current session.
Config get(fn config) config(): HostConfiguration<T::BlockNumber>;
/// Pending configuration (if any) for the next session.
PendingConfig: Option<HostConfiguration<T::BlockNumber>>;
}
}
decl_error! {
pub enum Error for Module<T: Trait> { }
}
decl_module! {
/// The parachains configuration module.
pub struct Module<T: Trait> for enum Call where origin: <T as system::Trait>::Origin {
type Error = Error<T>;
/// Set the validation upgrade frequency.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_validation_upgrade_frequency(origin, new: T::BlockNumber) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.validation_upgrade_frequency, new) != new
});
Ok(())
}
/// Set the validation upgrade delay.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_validation_upgrade_delay(origin, new: T::BlockNumber) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.validation_upgrade_delay, new) != new
});
Ok(())
}
/// Set the acceptance period for an included candidate.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_acceptance_period(origin, new: T::BlockNumber) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.acceptance_period, new) != new
});
Ok(())
}
/// Set the max validation code size for incoming upgrades.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_max_code_size(origin, new: u32) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.max_code_size, new) != new
});
Ok(())
}
/// Set the max head data size for paras.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_max_head_data_size(origin, new: u32) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.max_head_data_size, new) != new
});
Ok(())
}
/// Set the number of parathread execution cores.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_parathread_cores(origin, new: u32) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.parathread_cores, new) != new
});
Ok(())
}
/// Set the number of retries for a particular parathread.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_parathread_retries(origin, new: u32) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.parathread_retries, new) != new
});
Ok(())
}
/// Set the parachain validator-group rotation frequency
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_parachain_rotation_frequency(origin, new: T::BlockNumber) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.parachain_rotation_frequency, new) != new
});
Ok(())
}
/// Set the availability period for parachains.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_chain_availability_period(origin, new: T::BlockNumber) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.chain_availability_period, new) != new
});
Ok(())
}
/// Set the availability period for parathreads.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_thread_availability_period(origin, new: T::BlockNumber) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.thread_availability_period, new) != new
});
Ok(())
}
/// Set the scheduling lookahead, in expected number of blocks at peak throughput.
#[weight = (1_000, DispatchClass::Operational)]
pub fn set_scheduling_lookahead(origin, new: u32) -> DispatchResult {
ensure_root(origin)?;
Self::update_config_member(|config| {
sp_std::mem::replace(&mut config.scheduling_lookahead, new) != new
});
Ok(())
}
}
}
impl<T: Trait> Module<T> {
/// Called by the initializer to initialize the configuration module.
pub(crate) fn initializer_initialize(_now: T::BlockNumber) -> Weight {
0
}
/// Called by the initializer to finalize the configuration module.
pub(crate) fn initializer_finalize() { }
/// Called by the initializer to note that a new session has started.
pub(crate) fn initializer_on_new_session(_validators: &[ValidatorId], _queued: &[ValidatorId]) {
if let Some(pending) = <Self as Store>::PendingConfig::take() {
<Self as Store>::Config::set(pending);
}
}
fn update_config_member(
updater: impl FnOnce(&mut HostConfiguration<T::BlockNumber>) -> bool,
) {
let pending = <Self as Store>::PendingConfig::get();
let mut prev = pending.unwrap_or_else(Self::config);
if updater(&mut prev) {
<Self as Store>::PendingConfig::set(Some(prev));
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::mock::{new_test_ext, Initializer, Configuration, Origin};
use frame_support::traits::{OnFinalize, OnInitialize};
#[test]
fn config_changes_on_session_boundary() {
new_test_ext(Default::default()).execute_with(|| {
let old_config = Configuration::config();
let mut config = old_config.clone();
config.validation_upgrade_delay = 100;
assert!(old_config != config);
<Configuration as Store>::PendingConfig::set(Some(config.clone()));
Initializer::on_initialize(1);
assert_eq!(Configuration::config(), old_config);
assert_eq!(<Configuration as Store>::PendingConfig::get(), Some(config.clone()));
Initializer::on_finalize(1);
Configuration::initializer_on_new_session(&[], &[]);
assert_eq!(Configuration::config(), config);
assert!(<Configuration as Store>::PendingConfig::get().is_none());
})
}
#[test]
fn setting_pending_config_members() {
new_test_ext(Default::default()).execute_with(|| {
let new_config = HostConfiguration {
validation_upgrade_frequency: 100,
validation_upgrade_delay: 10,
acceptance_period: 5,
max_code_size: 100_000,
max_head_data_size: 1_000,
parathread_cores: 2,
parathread_retries: 5,
parachain_rotation_frequency: 20,
chain_availability_period: 10,
thread_availability_period: 8,
scheduling_lookahead: 3,
};
assert!(<Configuration as Store>::PendingConfig::get().is_none());
Configuration::set_validation_upgrade_frequency(
Origin::ROOT, new_config.validation_upgrade_frequency,
).unwrap();
Configuration::set_validation_upgrade_delay(
Origin::ROOT, new_config.validation_upgrade_delay,
).unwrap();
Configuration::set_acceptance_period(
Origin::ROOT, new_config.acceptance_period,
).unwrap();
Configuration::set_max_code_size(
Origin::ROOT, new_config.max_code_size,
).unwrap();
Configuration::set_max_head_data_size(
Origin::ROOT, new_config.max_head_data_size,
).unwrap();
Configuration::set_parathread_cores(
Origin::ROOT, new_config.parathread_cores,
).unwrap();
Configuration::set_parathread_retries(
Origin::ROOT, new_config.parathread_retries,
).unwrap();
Configuration::set_parachain_rotation_frequency(
Origin::ROOT, new_config.parachain_rotation_frequency,
).unwrap();
Configuration::set_chain_availability_period(
Origin::ROOT, new_config.chain_availability_period,
).unwrap();
Configuration::set_thread_availability_period(
Origin::ROOT, new_config.thread_availability_period,
).unwrap();
Configuration::set_scheduling_lookahead(
Origin::ROOT, new_config.scheduling_lookahead,
).unwrap();
assert_eq!(<Configuration as Store>::PendingConfig::get(), Some(new_config));
})
}
#[test]
fn non_root_cannot_set_config() {
new_test_ext(Default::default()).execute_with(|| {
assert!(Configuration::set_validation_upgrade_delay(Origin::signed(1), 100).is_err());
});
}
#[test]
fn setting_config_to_same_as_current_is_noop() {
new_test_ext(Default::default()).execute_with(|| {
Configuration::set_validation_upgrade_delay(Origin::ROOT, Default::default()).unwrap();
assert!(<Configuration as Store>::PendingConfig::get().is_none())
});
}
}
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! This module is responsible for maintaining a consistent initialization order for all other
//! parachains modules. It's also responsible for finalization and session change notifications.
//!
//! This module can throw fatal errors if session-change notifications are received after initialization.
use sp_std::prelude::*;
use frame_support::weights::Weight;
use primitives::{
parachain::{ValidatorId},
};
use frame_support::{
decl_storage, decl_module, decl_error,
};
use crate::{configuration, paras};
pub trait Trait: system::Trait + configuration::Trait + paras::Trait { }
decl_storage! {
trait Store for Module<T: Trait> as Initializer {
/// Whether the parachains modules have been initialized within this block.
///
/// Semantically a bool, but this guarantees it should never hit the trie,
/// as this is cleared in `on_finalize` and Frame optimizes `None` values to be empty values.
///
/// As a bool, `set(false)` and `remove()` both lead to the next `get()` being false, but one of
/// them writes to the trie and one does not. This confusion makes `Option<()>` more suitable for
/// the semantics of this variable.
HasInitialized: Option<()>;
}
}
decl_error! {
pub enum Error for Module<T: Trait> { }
}
decl_module! {
/// The initializer module.
pub struct Module<T: Trait> for enum Call where origin: <T as system::Trait>::Origin {
type Error = Error<T>;
fn on_initialize(now: T::BlockNumber) -> Weight {
// The other modules are initialized in this order:
// - Configuration
// - Paras
// - Scheduler
// - Inclusion
// - Validity
let total_weight = configuration::Module::<T>::initializer_initialize(now) +
paras::Module::<T>::initializer_initialize(now);
HasInitialized::set(Some(()));
total_weight
}
fn on_finalize() {
paras::Module::<T>::initializer_finalize();
configuration::Module::<T>::initializer_finalize();
HasInitialized::take();
}