Unverified Commit b042a5fb authored by Michael Müller's avatar Michael Müller Committed by GitHub

Add BTreeMap storage primitive (#284)

* Update readme

* Insert + Entry API

* Combine header + entries into one object

* Move insert logic into `Tree`

* Fix visibility + shorten code

* Add removal API (split/merge nodes) + quickcheck tests

* Simplify code, improve structure

* Add more API functions, improve code

* Update delegator instructions

* Style + Improve structure + Comments

* Clippy made me do it

* Improve code + tests + style

* Clarify search_linear

* Improve code structure and comments

* Improve code structure + add comments

* Fix style

* Fix style

* Improve comments

* Simplify code, improve comments

* Improve readability of tests

* Move expect to function where we can properly prove

* Fix style

* Move expect to function where we can properly prove

* Fix style

* Add debug_assert

* Improve correct_children_... code and docs

* Change kv tuple to named struct and simplify kv code

* Remove unsafe's, introduce indirection for accessing storage, revise pub's

* Put expensive tests behind ink-fuzz feature

* Improve structure

* Satisfy clippy, remove debugging code

* Use extend instead of append

* Adapt to new ink! env rev 3

* Fix comment

* Remove quickcheck tests (until issue with offchain API fixed)

* Refactoring and restructuring

* Refactoring and code restructuring

* Make clippy happy

* Fix metadata derive

* Fix metadata derive

* Add test for tinkering with balancedness

* Refactoring tests

* Update date and satisfy CI

* Improve docs

* Refactoring code

* Refactoring code

* Refactoring code

* Apply rustfmt

* Improve tests

* Apply suggestions from code review
Co-Authored-By: default avatarHero Bird <robin.freyler@gmail.com>

* Improve when/then structure

* Improve readability

* Rename pointer to index

* Rename internal nodes to branch nodes

* Rename entries to nodes

* Document loop behavior

* Improve unreachable! messages

* Improve comments + expect's

* Apply rustfmt

* Apply rustfmt
Co-authored-by: Hero Bird's avatarHero Bird <robbepop@web.de>
parent 2aabc9b6
Pipeline #79375 failed with stages
in 10 minutes and 41 seconds
......@@ -34,6 +34,9 @@ num-traits = { version = "0.2.1", default-features = false, features = ["i128"]
# Never use this crate outside of the off-chain environment!
rand = { version = "0.7", default-features = false, features = ["alloc"], optional = true }
[dev-dependencies]
itertools = "0.8.2"
[features]
default = ["std"]
std = [
......
This diff is collapsed.
// Copyright 2018-2020 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! A BTreeMap collection.
//!
//! This implementation follows the algorithm used by the Rust `BTreeMap` standard
//! library implementation. The Rust implementation was in general the blueprint for
//! this implementation. The major difference is that the Rust implementation is
//! in-memory, whereas this implementation uses the ink! primitives for storage.
//!
//! See https://github.com/rust-lang/rust/blob/master/src/liballoc/collections/btree
//! for the Rust implementation.
//!
//! The idea of a BTreeMap is to store many elements (i.e. key/value pairs)
//! in one tree node. Each of these elements can have a left and right child.
//! A simple node with three elements thus can look like this:
//!
//! ```no_compile
//! keys = [ key(a), key(b), key(c) ];
//! vals = [ value(a), value(b), value(c) ];
//! edges = [ 1, 2, 3, 4 ];
//! ```
//!
//! Here the left child of element `a` would be the node with the index `1`, its
//! right child the node with index `2`.
//!
//! This concept of multiple elements stored in one node suits our needs well,
//! since expensive storage fetches are reduced.
//!
//! For a description of the tree algorithm itself it's best to see the merge/split
//! method comments. A notable thing is that the algorithm will merge nodes if it's
//! possible to reduce storage space (see `handle_underfull_nodes()` for more info).
#[cfg(test)]
mod test_utils;
#[cfg(test)]
mod tests;
mod impls;
mod node;
mod search;
pub use self::impls::{
BTreeMap,
Entry,
};
This diff is collapsed.
// Copyright 2018-2019 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use self::SearchResult::{
Found,
NotFound,
};
use crate::storage::btree_map::{
impls::{
BTreeMap,
HandleType::{
Branch,
Leaf,
},
CAPACITY,
},
node::{
KVHandle,
Node,
NodeHandle,
},
};
use core::{
borrow::Borrow,
cmp::{
Ord,
Ordering,
},
};
use scale::Codec;
/// Result of a tree search.
pub(super) enum SearchResult {
/// Found the entry at the supplied position.
Found(KVHandle),
/// No search result, contains the position where an insert could be made.
NotFound(KVHandle),
}
/// Searches the tree for `key`.
///
/// If found returns `Found(pos)`
/// If not found returns `NotFound(last_pos_searched)`.
pub(super) fn search_tree<K, V, Q>(tree: &BTreeMap<K, V>, key: &Q) -> SearchResult
where
Q: Ord,
K: Ord + Borrow<Q> + Codec,
V: Codec,
{
let current_root = tree.root();
if tree.is_empty() || current_root.is_none() {
return NotFound(KVHandle::new(NodeHandle::new(0), 0))
}
let mut cur =
current_root.expect("we would already have returned if no root exists; qed");
loop {
let node = tree.get_node(cur).expect(
"node which is iterated over is either root or child node, \
but it always exists; qed",
);
match search_node(&node, tree.keys_in_node(cur), cur, key) {
Found(handle) => return Found(handle),
NotFound(handle) => {
match tree.get_handle_type(handle.node()) {
Leaf => return NotFound(handle),
Branch => {
// Go down then
cur = tree
.descend(handle)
.expect("a branch node always has a child; qed");
continue
}
}
}
}
}
}
/// Conducts a linear search for `key` in the elements contained in `node`.
///
/// If found returns `Found(pos)`
/// If not found returns `NotFound(last_pos_searched)`.
pub(super) fn search_node<K, V, Q>(
node: &Node<K, V>,
keys_in_node: [Option<&K>; CAPACITY],
node_handle: NodeHandle,
key: &Q,
) -> SearchResult
where
Q: Ord,
K: Borrow<Q>,
{
let iter = keys_in_node.iter().enumerate();
for (i, k) in iter {
match k {
None => return NotFound(KVHandle::new(node_handle, i)),
Some(node_key) => {
match key.cmp((**node_key).borrow()) {
Ordering::Greater => {}
Ordering::Equal => return Found(KVHandle::new(node_handle, i)),
Ordering::Less => return NotFound(KVHandle::new(node_handle, i)),
}
}
}
}
NotFound(KVHandle::new(node_handle, node.len()))
}
// Copyright 2018-2020 Parity Technologies (UK) Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::{
alloc::{
AllocateUsing,
BumpAlloc,
Initialize,
},
collections::btree_map::node::NodeHandle,
BTreeMap,
};
use ink_primitives::Key;
use itertools::Itertools;
/// Creates an empty map.
pub fn empty_map() -> BTreeMap<i32, i32> {
unsafe {
let mut alloc = BumpAlloc::from_raw_parts(Key([0x0; 32]));
BTreeMap::allocate_using(&mut alloc).initialize_into(())
}
}
/// Creates a map pre-filled with some key/value pairs.
pub fn filled_map() -> BTreeMap<i32, i32> {
let mut map = empty_map();
map.insert(5, 50);
map.insert(42, 420);
map.insert(1337, 13370);
map.insert(77, 770);
assert_eq!(map.len(), 4);
map
}
/// Returns all edges in the tree as one Vec.
pub fn all_edges(map: &BTreeMap<i32, i32>) -> Vec<u32> {
let mut v = Vec::new();
let mut processed_nodes = 0;
let mut node_index = 0;
loop {
if processed_nodes == map.node_count() {
break
}
// We iterate over all storage entities of the tree and skip vacant entities.
let handle = NodeHandle::new(node_index);
if let Some(node) = map.get_node(handle) {
let edges = node
.edges()
.to_vec()
.into_iter()
.filter_map(|x| x.map(|v| v.node()));
v.extend(edges);
processed_nodes += 1;
}
node_index += 1;
}
v
}
/// Returns `true` if every edge exists only once in the tree.
/// If duplicate edges are found each duplicate is printed to the console.
pub fn every_edge_exists_only_once(map: &BTreeMap<i32, i32>) -> bool {
let all_edges = all_edges(map);
let unique_edges: Vec<u32> = all_edges.clone().into_iter().unique().collect();
let only_unique_edges = all_edges.len() == unique_edges.len();
if !only_unique_edges {
unique_edges.iter().for_each(|x| {
if all_edges.iter().any(|a| *a == *x) {
eprintln!("duplicate {:?}", x);
}
});
}
only_unique_edges
}
/// Conducts repeated insert and remove operations into the map by iterating
/// over `xs`. For each odd number a defined number of insert operations
/// are executed. For each even number it's asserted that the previously
/// inserted elements are in the map and they are removed subsequently.
///
/// Using this scheme we get a sequence of insert and remove operations.
pub fn insert_and_remove(xs: Vec<i32>) {
let mut map = empty_map();
let mut count_inserts = 0;
let mut previous_even_x = None;
let number_inserts = 3;
xs.iter().for_each(|x| {
let x = *x;
if x % 2 == 0 {
// On even numbers we insert new nodes.
for a in x..x + number_inserts {
if let None = map.insert(a, a * 10) {
count_inserts += 1;
}
assert_eq!(map.len(), count_inserts);
}
previous_even_x = Some(x);
} else if x % 2 == 1 && previous_even_x.is_some() {
// if it's an odd number and we inserted in the previous run we assert
// that the insert worked correctly and remove the elements again.
let x = previous_even_x.unwrap();
for a in x..x + number_inserts {
assert_eq!(map.get(&a), Some(&(a * 10)));
assert_eq!(map.remove(&a), Some(a * 10));
assert_eq!(map.get(&a), None);
count_inserts -= 1;
assert_eq!(map.len(), count_inserts);
}
previous_even_x = None;
}
assert!(every_edge_exists_only_once(&map));
});
}
/// Asserts that there is no node in storage for the range `0..max_node_count`.
pub fn storage_empty(map: &BTreeMap<i32, i32>, max_node_count: u32) -> bool {
for i in 0..max_node_count {
assert!(map.get_node(NodeHandle::new(i)).is_none());
}
true
}
This diff is collapsed.
......@@ -14,6 +14,7 @@
pub mod binary_heap;
pub mod bitvec;
pub mod btree_map;
pub mod hash_map;
pub mod stash;
pub mod vec;
......@@ -86,6 +86,10 @@ pub use self::{
self,
BitVec,
},
btree_map::{
self,
BTreeMap,
},
hash_map::{
self,
HashMap,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment