Unverified Commit dffc8cf2 authored by Hero Bird's avatar Hero Bird Committed by GitHub

Implement storage (revision 2) module (#311)

* [core] apply rustfmt

* [core] fix warnings related to Wasm compilation

* [core] add SpreadLayout impl for DynamicAllocator

* [core] remove unused method on Bits256RefMut

* [core] apply rustfmt

* [core] remove some unneded ToDo comments

The ToDo comments have been moved to the associated PR description.

* [core] transit to new traits for LazyIndexMap

* [core] transit to new traits for storage::Vec

* [core] transit to new traits for storage::Stash

* [core] transit to new traits for storage::Bitvec

* [core] transit to new traits for dynamic storage allocator

* [core] transit to new traits for LazyHashMap

* [core] transit to new traits for storage::HashMap

* [core] apply rustfmt

* [core] remove old storage traits for storage::Pack

* [core] transit to new storage traits for LazyArray

* [core] transit to new storage traits for storage::SmallVec

* [core] transit to new storage traits for the rest of the lazy abstractions

* [core] transit to new storage traits for storage::Box

* [core] fix compile error in Drop impl for storage::Box

* [core] remove old storage trait impls for Bits256

* [core] remove old storage trait impls for dynamic storage allocator

* [core] apply rustfmt

* [core] remove old traits module

* [core] replace KeyPtr2 usage with KeyPtr

* [core] rename traits2 module to traits

* [core] apply rustfmt

* [core] add Drop impl to storage::Vec

* [core] don't clear storage if key is none for storage::Vec

* [core] impl Drop for storage::Stash

* [core] simplify trait bounds for LazyHashMap

* [core] impl Drop for storage::HashMap

* [core] add Drop impl for storage::SmallVec

* [core] add are_trait_objects lint as deny

* [core] fix minor formatting issue

* [core] add storage2::Memory utility

* [core] remove usage of storage::Pack from internals of storage::Bitvec

* [core] remove usage of storage::Pack from internals of storage::Stash

* [core] remove usage of storage::Pack from internals of storage::HashMap

* [core] add better Debug impl for LazyIndexMap

The improved impl shows the cached entries which were hidden in the old impl behind the UnsafeCell.

* [core] apply rustfmt

* [core] improve Debug impl for LazyHashMap

It now displays its internal cached entries.

* [core] improve Debug impl for lazy::Entry

* [core] improve Debug impl for LazyCell

* [core] improve Debug impl for LazyArray

* [core] apply rustfmt

* [core] add REQUIRES_DEEP_CLEAN_UP to SpreadLayout

With this we have a way for types to perform an optimized storage clean-up without having to load them in case they do not require a deep clean-up of their state.

* [core] implement REQUIRES_DEEP_CLEAN_UP for all built-in types

* [core] add non-storage trait impls for storage::HashMap

* [core] simplify traits bounds of SpreadLayout impl for storage::HashMap

* [core] fix bug in Wasm compilation

* [core] add initial unit tests for storage::HashMap

* [core] allow loading without key from LazyHashMap

* [core] merge storage::HashMap insert and insert_get and fix bugs with it

* [core] add new unit test for storage::HashMap

* [core] generally allow lazy loading without keys for lazy abstractions

* [core] apply rustfmt

* [core] remove outdated docs of storage::HashMap::insert

* [core] add unit test for storage::HashMap::contains_key

* [core] apply rustfmt to storage::HashMap unit tests

* [core] add unit test for storage::HashMap::{get, get_mut}

* [core] fix the doc comment of storage::HashMap::remove

* [core] add unit test for storage::HashMap::take

* [core] add unit test for storage::HashMap::insert

* [core] remove storage::HashMap::remove

The API cannot be implemented properly at this point.

* [core] implement Iterator::count efficiently for storage::HashMap iterators

* [core] add prelude trait impls for crypto hashers

* [core] add unit test for storage::HashMap::iter

* [core] remove outdated doc comment line

* [core] add doc comments to fowarding-to-packed utility functions

* [core] add some high-level documentation for some root storage2 modules

* [core] add some more high-level docs

* [core] add return value to storage::Stash::defrag

The returned value tells the caller how many storage cells have actually been freed by the routine.

* [core] add return value to storage::HashMap::defrag

* [core] add unit test for storage::HashMap::{values, values_mut}

Also add tests for Iterator::size_hint impls.

* [core] add tests for Iterator::size_hint impls of storage::Vec

* [core] add unit test for storage::HashMap::keys

* [core] add unit test for storage::HashMap::defrag

* [core] add unit tests for LazyIndexMap

* [core] remove lazy::Entry::take_value

* [core] remove LazyIndexMap::take

* [core] remove Entry::set_state

Uses have been replaced with Entry::replace_state.

* [core] remove Lazy{Array, HashMap}::take method

Replace uses with Lazy{Array, HashMap}::put_get(.., None)

* [core] add unit test for LazyIndexMap::put

* [core] add unit test for LazyIndexMap::swap

* [core] apply rustfmt

* [core] cover Default impl of LazyIndexMap with unit test

* [core] move imports to top for LazyIndexMap .rs file

* [core] refactor lazy::Entry internals a bit

* [core] add unit tests for Index impl of storage::Vec

* [core] add unit tests for Index impl of storage::SmallVec

* [core] add tests for Index impl of StorageStash

* [core] improve panic message for Index{Mut} impl of storage::Stash

* [core] add unit tests for Index{Mut} impl of storage::Stash

* [core] extend unit test for storage::Stash::get

* [core] disable certain tests in --release mode testing

* [core] add unit test for LazyIndexMap::{get, get_mut}

* [core] add some unit tests for LazyArray

* [core] add some more unit tests for LazyArray

* [core] add some more unit tests to LaryArray

* [core] apply rustfmt

* [core] add unit tests for LazyCell

* [core] add unit test for SpreadLayout impl of LazyCell

* [core] extend SpreadLayout test for LazyCell

* [core] extend SpreadLayout test to also cover the clear_spread impl

* [core] rename unit test for LazyCell

* [core] fix clippy warning

* [core] fix some LazyCell cache entry in lazy form

* [core] add new unit test for Debug impl of lazy initialized LazyCell

* [core] add more unit tests for lazily initialized LazyCell

* [core] implement shallow clean-up of storage via LazyCell

* [core] test that a lazily loaded LazyCell preserves its cached value

* [core] apply rustfmt

* [core] add additional check for LazyCell cache preservation

* [core] fix bug in LazyIndexMap::clear_packed_at

* [core] add unit test for SpreadLayout impl of LazyIndexMap

* [core] fix bug in LazyArray::clear_packed_at

* [core] add unit test for SpreadLayout impl of LazyArray

* [core] make LazyArray::capacity and SmallVec::capcity more user friendly

* [core] remove unnecessary trait bounds

* [core] remove more unnecessary trait bounds

* [core] add initial unit test for LazyHashMap

* [core] add unit test for LazyHashMap::key_at

* [core] apply rustfmt

* [core] indent a block in test

* [core] add unit test for LazyHashMap::put_get

* [core] add unit test for LazyHashMap::{get, get_mut}

* [core] add unit test for LazyHashMap::put

* [core] add unit test for LazyHashMap::swap

* [core] make hash builders reset their accumulator upon finalization

* [core] add unit test for SpreadLayout impl of LazyHashMap

* [core] fix unit test for LazyHashMap::key_at

Also add prefix to hash-key calculation.

* [core] add unit tests for SpreadLayout impl of storage::Vec

* [core] add unit tests for SpreadLayout impl of storage::SmallVec

* [core] add unit tests for SpreadLayout impl of storage::Stash

* [core] apply rustfmt

* [core] add unit tests for SpreadLayout impl of storage::HashMap

* [core] add unit test for DynamicAllocation::key

* [core] add unit tests for SpreadLayout impl of storage::Bitvec

* [core] fix LazyCell::get unit test

* [core] remove unused dependencies from Cargo.toml

* [core] add missing docs for storage::{Stash, HashMap}

* [core] deny missing docs of public items

* [core] add Debug impl to storage::Box

* [core] add unit tests for storage::Box

* [core] remove internal Pack::{get, get_mut} methods

* [core] fix bug in storage::Memory::{get, get_mut} API

* [core] add unit tests for storage::Pack

* [core] improve storage::Pack unit tests

* [core] experimental inline(never) for debug_assertions compilation

* [core] apply rustfmt

* [core] remove experimental #[inline(never)]

* [core] add unit test for Default impl of storage::Pack

* [core] add unit tests for storage::Memory

* [core] fix a unit test for storage::Box

The storage::Box tests did not reset the dynamic storage allocator instance in between their runs which caued them to have side effects on to each other if run single threaded.

* [core] fix minor bug in BitRefMut utility of storage::Bitvec

* [core] cover storage::Bitvec::get_mut in get_works unit test

* [core] add unit tests for BitRefMut utility of storage::Bitvec

* [core] apply rustfmt

* [core] improve panic message when encountering a double free

* [core] adjust double free unit test for storage::Box

* [core] improve double free of dynamic storage panic message

* [core] apply rustfmt

* [core] merge Bits256Ref and Bits256RefMut into ChunkRef<T>

* [core] split access.rs into bitref.rs and bitsref.rs

* [core] apply rustfmt

* [core] replace transmute with pointer cast

Thanks clippy!

* [core] add comment to explain repr(C)

* [core] add PartialEq and Eq impls to BitRefMut

* [core] add unit tests for ChunkRef

* [core] add failure unit tests for dynamic storage allocator

* [core] fix bug in SpreadLayout impl of Option<T>

* [core] add unit test for dynamic storage allocator SpreadLayout impl

* [core] fix SpreadLayout impl for Result<T, E>

* [core] fix yet another bug in SpreadLayout impl of Result<T, E>

* [core] move forward_supported_array_lens macro to usage site

* [core] refactor some code duplication with clear_spread_root_opt

* [core] fix doc comment in storage::Pack

* [core] remove some unused unsafe blocks

They are going to be re-introduced once the unsafe_op_in_unsafe_fn lint has been implemented in the Rust compiler.

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* [core] remove usage of storage::Pack in dynamic storage allocator

* [core] improve panic message in Lazy::{get, get_mut}

* [core] add test for SpreadLayout::clear_spread impl of dynamic storage alloc

* [core] remove code dupe

* [core] refactor clear_spread_root_opt utility function

* [core] implement SpreadLayout::REQUIRES_DEEP_CLEAN_UP for some types

* [core] move from bool to u8 for Option and Result SpreadLayout impls

* [core] fix bug in SpreadLayout impl for Option

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* [core] update LazyCell SAFETY comment

* [core] update Entry docs

* [core] remove unneeded code in lazy::Entry::pull_packed_root

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* [core] remove commented out code

* [core] add new unit test for dynamic storage allocator

* [core] refactor global storage allocator initialization routines

* [core] fix Wasm compilation errors

* [core] apply rustfmt

* [core] surpress bad clippy lint

* [core] remove dead code

* [core] improve call_setup_works test

* [core] fix bug in initialize_for for off-chain env

* [core] initial steps to factor out BitStash from DynamicAllocator

* [core] apply rustfmt

* [core] add Derive impl for BitStash

* [core] make use of storage::BitStash from dynamic storage allocator

* [core] add unit tests for storage::BitStash

* [core] apply rustfmt

* [core] remove invalid TODO comment

* [core] fix some out of bounds panic messages

* [core] remove deliberate memory leak in test suite

* [core] fix build failure for Wasm target

* [core] add unit tests for SpreadLayout & PackedLayout impls of primitives

* [core] add unit tests for packed layout explicitely

* Fix some typos

* Add simple double ended iter test

* typos

* comment typos

* split hashmap to hash map in comments

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo in unreachable! message
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo in expects message
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* [core] add more comments to storage2::HashMap::defrag

* [core] make early return for storage2::HashMap::defrag for limit = 0

* [core] improve storage2::HashMap::contains_key implementation

* [core] rename new_vec_works test to new_works

* [core] apply Andrew's suggestions (and more)

* [core] fix typo: increase -> decrease

* [core] add panic to Bitvec::push in case it reached its maximum capacity

* [core] update comments for storage bit stash

* [core] add more explanation comments

* [core] some more renamings of test internals

* improve reasoning
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>

* fix typo
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>
Co-authored-by: Andrew Jones's avatarAndrew Jones <ascjones@gmail.com>
parent 18085509
Pipeline #93406 passed with stages
in 8 minutes and 16 seconds
......@@ -16,6 +16,8 @@
[f2]: https://paritytech.github.io/ink/ink_core
[h1]: https://img.shields.io/badge/docs-abi-blue.svg
[h2]: https://paritytech.github.io/ink/ink_abi
[i1]: https://img.shields.io/badge/docs-prelude-blue.svg
[i2]: https://paritytech.github.io/ink/ink_prelude
**IMPORTANT NOTE:** WORK IN PROGRESS! Do not expect this to be working.
......@@ -25,9 +27,9 @@ For more information please visit [the ink! tutorial](https://substrate.dev/subs
## Developer Documentation
| `ink_abi` | `ink_core` |
| ------------- | ------------- |
| [![][h1]][h2] | [![][f1]][f2] |
| `ink_abi` | `ink_core` | `ink_prelude` |
| ------------- | ------------- | ------------- |
| [![][h1]][h2] | [![][f1]][f2] | [![][i1]][i2] |
### Interaction with Substrate
......
......@@ -23,13 +23,13 @@ ink_prelude = { version = "2.1.0", path = "../prelude/", default-features = fals
scale = { package = "parity-scale-codec", version = "1.3", default-features = false, features = ["derive", "full"] }
derive_more = { version = "0.99", default-features = false, features = ["from", "display"] }
smallvec = { version = "1.2", default-features = false, features = ["union"] }
cfg-if = "0.1"
num-traits = { version = "0.2", default-features = false, features = ["i128"] }
cfg-if = "0.1"
array-init = "0.1"
generic-array = "0.14.1"
paste = "0.1"
# Hashes for the off-chain environment.
byteorder = { version = "1.3", optional = true }
blake2-rfc = { version = "0.2", optional = true }
sha2 = { version = "0.8", optional = true }
tiny-keccak = { version = "2.0", optional = true }
......@@ -63,7 +63,6 @@ std = [
"rand/std",
"num-traits/std",
# Enables hashing crates for off-chain environment.
"byteorder",
"blake2-rfc",
"sha2",
"tiny-keccak",
......
......@@ -18,6 +18,8 @@ mod builder;
mod instantiate;
mod utils;
/// The compile-time states of builder for calls and instantiations.
#[doc(hidden)]
pub mod state {
pub use crate::env::call::{
instantiate::state::{
......
......@@ -257,11 +257,17 @@ pub struct DefaultAccounts<T>
where
T: EnvTypes,
{
/// The predefined `ALICE` account holding substantial amounts of value.
pub alice: T::AccountId,
/// The predefined `BOB` account holding some amounts of value.
pub bob: T::AccountId,
/// The predefined `CHARLIE` account holding some amounts of value.
pub charlie: T::AccountId,
/// The predefined `DJANGO` account holding no value.
pub django: T::AccountId,
/// The predefined `EVE` account holding no value.
pub eve: T::AccountId,
/// The predefined `FRANK` account holding no value.
pub frank: T::AccountId,
}
......
......@@ -111,8 +111,8 @@ pub trait Finalize<H>
where
H: Hasher,
{
fn finalize_using(&self, output: &mut <H as Hasher>::Output);
fn finalize(&self) -> <H as Hasher>::Output;
fn finalize_using(&mut self, output: &mut <H as Hasher>::Output);
fn finalize(&mut self) -> <H as Hasher>::Output;
}
impl<H, S> Finalize<H> for HashBuilder<H, S>
......@@ -120,11 +120,13 @@ where
H: Hasher,
S: Accumulator,
{
fn finalize_using(&self, output: &mut <H as Hasher>::Output) {
<H as Hasher>::finalize_immediate(self.strategy.as_slice(), output)
fn finalize_using(&mut self, output: &mut <H as Hasher>::Output) {
let output = <H as Hasher>::finalize_immediate(self.strategy.as_slice(), output);
self.strategy.reset();
output
}
fn finalize(&self) -> <H as Hasher>::Output {
fn finalize(&mut self) -> <H as Hasher>::Output {
let mut output = <<H as Hasher>::Output as Default>::default();
Self::finalize_using(self, &mut output);
output
......
......@@ -34,6 +34,7 @@ macro_rules! impl_hasher_for {
struct $ty_name:ident($fn_name:ident, $output_len:literal);
) => {
$( #[$doc] )*
#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum $ty_name {}
impl Hasher for $ty_name {
......@@ -47,21 +48,17 @@ macro_rules! impl_hasher_for {
}
impl_hasher_for! {
/// SHA2 256-bit hasher.
#[derive(Debug)]
struct Sha2x256Hasher(sha2_256, 32);
}
impl_hasher_for! {
/// KECCAK 256-bit hasher.
#[derive(Debug)]
struct Keccak256Hasher(keccak_256, 32);
}
impl_hasher_for! {
/// BLAKE2 256-bit hasher.
#[derive(Debug)]
struct Blake2x256Hasher(blake2_256, 32);
}
impl_hasher_for! {
/// BLAKE2 128-bit hasher.
#[derive(Debug)]
struct Blake2x128Hasher(blake2_128, 16);
}
......@@ -22,7 +22,9 @@
#![cfg_attr(not(feature = "std"), no_std)]
#![deny(
missing_docs,
bad_style,
bare_trait_objects,
const_err,
improper_ctypes,
non_shorthand_field_patterns,
......@@ -49,6 +51,7 @@ extern crate ink_alloc;
pub mod env;
pub mod hash;
pub mod storage;
pub mod storage2;
// Needed for derive macros of `core/derive` sub crate.
pub(crate) use crate as ink_core;
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::hash::{
Blake2x256,
Wrap,
};
use ink_primitives::Key;
/// A unique dynamic allocation.
///
/// This can refer to a dynamically allocated storage cell.
/// It has been created by a dynamic storage allocator.
/// The initiator of the allocation has to make sure to deallocate
/// this dynamic allocation again using the same dynamic allocator
/// if it is no longer in use.
///
/// # Note
///
/// Normally instances of this type are not used directly and instead
/// a [`storage::Box`](`crate::storage2::Box`) is used instead.
#[derive(
Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, scale::Encode, scale::Decode,
)]
pub struct DynamicAllocation(pub(super) u32);
impl DynamicAllocation {
/// Returns the allocation identifier as `u32`.
pub(super) fn get(self) -> u32 {
self.0
}
/// Returns the storage key associated with this dynamic allocation.
pub fn key(self) -> Key {
// We create a 25-bytes buffer for the hashing.
// This is due to the fact that we prepend the `u32` encoded identifier
// with the `b"DYNAMICALLY ALLOCATED"` byte string which has a length
// 21 bytes. Since `u32` always has an encoding length of 4 bytes we
// end up requiring 25 bytes in total.
// Optimization Opportunity:
// Since ink! always runs single threaded we could make this buffer
// static and instead reuse its contents with every invocation of this
// method. However, this would introduce `unsafe` Rust usage.
#[rustfmt::skip]
let mut buffer: [u8; 25] = [
b'D', b'Y', b'N', b'A', b'M', b'I', b'C', b'A', b'L', b'L', b'Y',
b' ',
b'A', b'L', b'L', b'O', b'C', b'A', b'T', b'E', b'D',
b'_', b'_', b'_', b'_',
];
// Encode the `u32` identifier requires a 4 bytes buffer.
let mut hash_buffer = Wrap::from(&mut buffer[21..25]);
<u32 as scale::Encode>::encode_to(&self.0, &mut hash_buffer);
let mut output = [0x00_u8; 32];
<Blake2x256>::hash_bytes_using(&buffer, &mut output);
Key::from(output)
}
}
#[test]
fn get_works() {
let expected_keys = [
b"\
\x0A\x0F\xF5\x30\xBD\x5A\xB6\x67\
\x85\xC9\x74\x6D\x01\x33\xD7\xE1\
\x24\x40\xC4\x67\xA9\xF0\x6D\xCA\
\xE7\xED\x2E\x78\x32\x77\xE9\x10",
b"\
\x11\x5A\xC0\xB2\x29\xA5\x34\x10\
\xB0\xC0\x2D\x47\x49\xDC\x7A\x09\
\xB9\x6D\xF9\x51\xB6\x1D\x4F\x3B\
\x4E\x75\xAC\x3B\x14\x57\x47\x96",
];
assert_eq!(DynamicAllocation(0).key(), Key(*expected_keys[0]));
assert_eq!(DynamicAllocation(1).key(), Key(*expected_keys[1]));
}
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::DynamicAllocation;
use crate::storage2::{
collections::BitStash,
traits::{
KeyPtr,
SpreadLayout,
},
};
/// The dynamic allocator.
///
/// Manages dynamic storage allocations in a very efficient and economic way.
#[derive(Debug, Default, PartialEq, Eq)]
pub struct DynamicAllocator {
allocations: BitStash,
}
impl SpreadLayout for DynamicAllocator {
const FOOTPRINT: u64 = <BitStash as SpreadLayout>::FOOTPRINT;
fn pull_spread(ptr: &mut KeyPtr) -> Self {
Self {
allocations: SpreadLayout::pull_spread(ptr),
}
}
fn push_spread(&self, ptr: &mut KeyPtr) {
SpreadLayout::push_spread(&self.allocations, ptr);
}
fn clear_spread(&self, ptr: &mut KeyPtr) {
SpreadLayout::clear_spread(&self.allocations, ptr);
}
}
impl DynamicAllocator {
/// Returns a new dynamic storage allocation.
///
/// # Panics
///
/// If the dynamic allocator ran out of free dynamic allocations.
pub fn alloc(&mut self) -> DynamicAllocation {
DynamicAllocation(self.allocations.put())
}
/// Frees the given dynamic storage allocation.
///
/// This makes the given dynamic storage allocation available again
/// for new dynamic storage allocations.
///
/// # Panics
///
/// Panics if the given dynamic allocation is invalid.
/// A dynamic allocation is invalid if it is not represented as occupied
/// in the `free` list.
pub fn free(&mut self, allocation: DynamicAllocation) {
let index = allocation.get();
if !self
.allocations
.take(index)
.expect("invalid dynamic storage allocation")
{
panic!(
"encountered double free of dynamic storage: at index {}",
index
)
}
}
}
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::DynamicAllocator;
use crate::storage2::traits::pull_spread_root;
use cfg_if::cfg_if;
use ink_primitives::Key;
/// The default dynamic allocator key offset.
///
/// This is where the dynamic allocator is stored on the contract storage.
const DYNAMIC_ALLOCATOR_KEY_OFFSET: [u8; 32] = [0xFE; 32];
/// The phase in which a contract execution can be.
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub enum ContractPhase {
/// Initializes the global dynamic storage allocator from scratch.
///
/// Upon initialization it will be created from scratch as if the
/// contract has been deployed for the first time.
Deploy,
/// Initializes the global dynamic storage allocator from storage.
///
/// Upon initialization the dynamic storage allocator will be pulled
/// from the contract storage with the assumption that a former
/// contract deployment has already taken place in the past.
Call,
}
/// The state of the dynamic allocator global instance.
#[derive(Debug)]
#[allow(clippy::large_enum_variant)]
enum DynamicAllocatorState {
/// The global instance has not yet been initialized.
///
/// Upon initialization it will be created from scratch as if the
/// contract has been deployed for the first time.
UninitDeploy,
/// The global instance has not yet been initialized.
///
/// Upon initialization it will be pulled from the contract storage
/// with the assumption that a former contract deployment has already
/// taken place in the past.
UninitCall,
/// The global instance has already been initialized successfully.
Initialized(DynamicAllocator),
}
impl From<ContractPhase> for DynamicAllocatorState {
fn from(phase: ContractPhase) -> Self {
match phase {
ContractPhase::Deploy => DynamicAllocatorState::UninitDeploy,
ContractPhase::Call => DynamicAllocatorState::UninitCall,
}
}
}
cfg_if! {
if #[cfg(all(not(feature = "std"), target_arch = "wasm32"))] {
// Procedures for the Wasm compilation:
/// The global instance for the dynamic storage allocator.
static mut GLOBAL_INSTANCE: DynamicAllocatorState = DynamicAllocatorState::UninitDeploy;
/// Commands the (re-)initialization of the global instance for the dynamic
/// storage allocator.
pub fn initialize_for(phase: ContractPhase) {
let instance = unsafe { &mut GLOBAL_INSTANCE };
// We do not allow reinitialization for Wasm targets for performance reasons.
if let DynamicAllocatorState::Initialized(_) = instance {
panic!("cannot reinitialize dynamic storage allocator instance in Wasm");
}
*instance = phase.into();
}
/// Runs the given closure on the global instance for the dynamic storage allocator.
pub fn on_call<F, R>(f: F) -> R
where
F: FnOnce(&mut DynamicAllocator) -> R,
{
let instance = unsafe { &mut GLOBAL_INSTANCE };
match instance {
DynamicAllocatorState::UninitDeploy => {
let mut allocator = DynamicAllocator::default();
let result = f(&mut allocator);
*instance = DynamicAllocatorState::Initialized(allocator);
result
}
DynamicAllocatorState::UninitCall => {
let mut allocator = pull_spread_root::<DynamicAllocator>(&Key(DYNAMIC_ALLOCATOR_KEY_OFFSET));
let result = f(&mut allocator);
*instance = DynamicAllocatorState::Initialized(allocator);
result
}
DynamicAllocatorState::Initialized(ref mut allocator) => {
f(allocator)
}
}
}
} else if #[cfg(feature = "std")] {
// Procedures for the off-chain environment and testing compilation:
use ::core::cell::RefCell;
thread_local!(
/// The global instance for the dynamic storage allocator.
static GLOBAL_INSTANCE: RefCell<DynamicAllocatorState> = RefCell::new(DynamicAllocatorState::UninitDeploy);
);
/// Commands the (re-)initialization of the global instance for the dynamic
/// storage allocator.
pub fn initialize_for(phase: ContractPhase) {
GLOBAL_INSTANCE.with(|instance| {
instance.replace_with(|_| phase.into())
});
}
/// Runs the given closure on the global instance for the dynamic storage allocator.
pub fn on_call<F, R>(f: F) -> R
where
F: FnOnce(&mut DynamicAllocator) -> R,
{
GLOBAL_INSTANCE.with(|instance| {
match &mut *instance.borrow_mut() {
instance @ DynamicAllocatorState::UninitDeploy => {
let mut allocator = DynamicAllocator::default();
let result = f(&mut allocator);
*instance = DynamicAllocatorState::Initialized(allocator);
result
}
instance @ DynamicAllocatorState::UninitCall => {
let mut allocator = pull_spread_root::<DynamicAllocator>(&Key(DYNAMIC_ALLOCATOR_KEY_OFFSET));
let result = f(&mut allocator);
*instance = DynamicAllocatorState::Initialized(allocator);
result
}
DynamicAllocatorState::Initialized(instance) => {
f(instance)
}
}
})
}
} else {
compile_error! {
"ink! only support compilation as `std` or `no_std` + `wasm32-unknown`"
}
}
}
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! The default dynamic storage allocator.
//!
//! Allows to allocate storage cells in a dynamic fashion.
//! This is important if users want to combine types of varying storage
//! footprints. For example, dynamic allocations are required whenever
//! a user wants to use a storage collection (e.g. `storage::Vec`) in
//! another storage collection: `storage::Vec<storage::Vec<T>>`
//!
//! # Simplification
//!
//! The contracts pallet is using 256 bit keys for identifying storage cells.
//! This implies a storage space of 2^256 cells which is big enough to say that
//! there are probably never going to happen collisions anywhere at any time
//! if keys are chosen randomly. Using the built-in crypto hashers on unique
//! input we can be sure that there are never going to be collisions in this
//! space of 2^256 cells.
//!
//! This way we can reduce the problem of finding another region in our storage
//! that fits certain requirements (e.g. a minimum size) to the problem of
//! finding another uniform slot. Since we are on 32-bit WebAssembly we have
//! memory limitations that makes it impractical to have more than 2^32 dynamic
//! allocated entities and so we can create another limitation for having a
//! total of 2^32 dynamic allocations at any point in time.
//! This enables us to have 32-bit keys instead of 256-bit keys.
//!
//! We can convert such 32-bit keys (represented by e.g. a `u32`) into 256-bit
//! keys by using one of the built-in crypto hashes that has a 256-bit output,
//! e.g. KECCAK, SHA2 or BLAKE2. For technical reasons we should prepend the
//! bytes of the 32-bit key by some unique byte sequence, e.g.:
//! ```no_compile
//! let key256 = blake2x256(b"DYNAMICALLY ALLOCATED", bytes(key32));
//! ```
//!
//! # Internals
//!
//! As described in [# Simplification] there are 2^32 possible uniform dynamic
//! allocations available. For each such slot the dynamic allocator stores via
//! a single bit in a bitvector if that slot is free or occupied.
//! This bitvector is called the `free` list.
//! However, searching in this `free` list for a 0 bit and thus a free slot
//! for a dynamic allocation would mean that for every 256 consecutively
//! occupied dynamic allocations there was a contract storage lookup required.
//! This might seem a lot but given that there could be thousands or
//! tens of thousands of dynamic allocations at any given time this might not scale
//! well.
//! For the reason of improving scalability we added another vector: the
//! so-called `set_bits` vector.
//! In this vector every `u8` element densely stores the number of set bits
//! (bits that are `1` or `true`) for each 256-bit package in the `free` list.
//! (Note that the `free` list is organized in 256-bit chunks of bits.)
//!
//! This way, to search for an unoccupied dynamic allocation we iterate over
//! the set-bits vector which is 32 times more dense than our `free` list.
//! The additional density implies that we can query up to 8192 potential
//! dynamic storage allocations with a single contract storage look-up.
mod allocation;
mod allocator;
mod init;
#[cfg(test)]
mod tests;
pub use self::{
allocation::DynamicAllocation,
init::{
initialize_for,
ContractPhase,
},
};
use self::{
allocator::DynamicAllocator,
init::on_call,
};
/// Returns a new dynamic storage allocation.
pub fn alloc() -> DynamicAllocation {
on_call(DynamicAllocator::alloc)
}
/// Frees the given dynamic storage allocation.
///
/// This makes the given dynamic storage allocation available again
/// for new dynamic storage allocations.
pub fn free(allocation: DynamicAllocation) {
on_call(|allocator| allocator.free(allocation))
}
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.