Commit ca696fdc authored by Gautam Dhameja's avatar Gautam Dhameja Committed by Bastian Köcher
Browse files

Documentation for Parity SCALE Codec (#94)

* Documentation for Parity SCALE Codec.

* addressed review comments on docs

* added example for CompactAs and fixed a test

* minor fixes
parent 132e6479
Pipeline #39704 passed with stages
in 10 minutes and 11 seconds
= Codec
.Summary
[source, toml]
----
include::Cargo.toml[lines=2..5]
----
.Description
----
include::src/lib.rs[tag=description]
----
# Parity SCALE Codec
Rust implementation of the SCALE (Simple Concatenated Aggregate Little-Endian) data format for types used in the Parity Substrate framework.
<!-- Inspired from Gav's codec overview written for Subtrate docs site -->
SCALE is a light-weight format which allows encoding (and decoding) which makes it highly suitable for resource-constrained execution environments like blockchain runtimes and low-power, low-memory devices.
It is important to note that the encoding context (knowledge of how the types and data structures look) needs to be known separately at both encoding and decoding ends. The encoded data does not include this contextual information.
To get a better understanding of how the encoding is done for different types, take a look at the [low-level data formats overview page at the Substrate docs site](https://docs.substrate.dev/docs/low-level-data-formats).
## Implementation
The codec is implemented using the following traits:
### Encode
The `Encode` trait is used for encoding of data into the SCALE format. The `Encode` trait contains the following functions:
* `size_hint(&self) -> usize`: Gets the capacity (in bytes) required for the encoded data. This is to avoid double-allocation of memory needed for the encoding. It can be an estimate and does not need to be an exact number. If the size is not known, even no good maximum, then we can skip this function from the trait implementation. This is required to be a cheap operation, so should not involve iterations etc.
* `encode_to<T: Output>(&self, dest: &mut T)`: Encodes the value and appends it to a destination buffer.
* `encode(&self) -> Vec<u8>`: Encodes the type data and returns a slice.
* `using_encoded<R, F: FnOnce(&[u8]) -> R>(&self, f: F) -> R`: Encodes the type data and executes a closure on the encoded value. Returns the result from the executed closure.
**Note:** Implementations should override `using_encoded` for value types and `encode_to` for allocating types. `size_hint` should be implemented for all types, wherever possible. Wrapper types should override all methods.
### Decode
The `Decode` trait is used for deserialization/decoding of encoded data into the respective types.
* `fn decode<I: Input>(value: &mut I) -> Result<Self, Error>`: Tries to decode the value from SCALE format to the type it is called on. Returns an `Err` if the decoding fails.
### CompactAs
The `CompactAs` trait is used for wrapping custom types/structs as compact types, which makes them even more space/memory efficient. The compact encoding is described [here](https://docs.substrate.dev/docs/low-level-data-formats#section-compactgeneral-integers).
* `encode_as(&self) -> &Self::As`: Encodes the type (self) as a compact type. The type `As` is defined in the same trait and its implementation should be compact encode-able.
* `decode_from(_: Self::As) -> Self`: Decodes the type (self) from a compact encode-able type.
### HasCompact
The `HasCompact` trait, if implemented, tells that the corresponding type is a compact encode-able type.
## Usage Examples
Following are some examples to demonstrate usage of the codec.
### Simple types
```rust
#[derive(Debug, PartialEq, Encode, Decode)]
enum EnumType {
#[codec(index = "15")]
A,
B(u32, u64),
C {
a: u32,
b: u64,
},
}
let a = EnumType::A;
let b = EnumType::B(1, 2);
let c = EnumType::C { a: 1, b: 2 };
a.using_encoded(|ref slice| {
assert_eq!(slice, &b"\x0f");
});
b.using_encoded(|ref slice| {
assert_eq!(slice, &b"\x01\x01\0\0\0\x02\0\0\0\0\0\0\0");
});
c.using_encoded(|ref slice| {
assert_eq!(slice, &b"\x02\x01\0\0\0\x02\0\0\0\0\0\0\0");
});
let mut da: &[u8] = b"\x0f";
assert_eq!(EnumType::decode(&mut da).ok(), Some(a));
let mut db: &[u8] = b"\x01\x01\0\0\0\x02\0\0\0\0\0\0\0";
assert_eq!(EnumType::decode(&mut db).ok(), Some(b));
let mut dc: &[u8] = b"\x02\x01\0\0\0\x02\0\0\0\0\0\0\0";
assert_eq!(EnumType::decode(&mut dc).ok(), Some(c));
let mut dz: &[u8] = &[0];
assert_eq!(EnumType::decode(&mut dz).ok(), None);
```
### Compact type with HasCompact
```rust
#[derive(Debug, PartialEq, Encode, Decode)]
struct Test1CompactHasCompact<T: HasCompact> {
#[codec(compact)]
bar: T,
}
#[derive(Debug, PartialEq, Encode, Decode)]
struct Test1HasCompact<T: HasCompact> {
#[codec(encoded_as = "<T as HasCompact>::Type")]
bar: T,
}
let test_val: (u64, usize) = (0u64, 1usize);
let encoded = Test1HasCompact { bar: test_val.0 }.encode();
assert_eq!(encoded.len(), test_val.1);
assert_eq!(<Test1CompactHasCompact<u64>>::decode(&mut &encoded[..]).unwrap().bar, test_val.0);
```
### Type with CompactAs
```rust
#[cfg_attr(feature = "std", derive(Serialize, Deserialize, Debug))]
#[derive(PartialEq, Eq, Clone)]
struct StructHasCompact(u32);
impl CompactAs for StructHasCompact {
type As = u32;
fn encode_as(&self) -> &Self::As {
&12
}
fn decode_from(_: Self::As) -> Self {
StructHasCompact(12)
}
}
impl From<Compact<StructHasCompact>> for StructHasCompact {
fn from(_: Compact<StructHasCompact>) -> Self {
StructHasCompact(12)
}
}
#[derive(Debug, PartialEq, Encode, Decode)]
enum TestGenericHasCompact<T> {
A {
#[codec(compact)] a: T
},
}
let a = TestGenericHasCompact::A::<StructHasCompact> {
a: StructHasCompact(12325678),
};
let encoded = a.encode();
assert_eq!(encoded.len(), 2);
```
## Derive macros
This repository also contains an implementation of derive macros for the Parity SCALE codec. These macros can be used to implement the encode and decode functions for types using attributes. For reference, see the `derive` attributes on the types defined in the examples above.
## License
This Rust implementation of Parity SCALE Codec is licenced under the [Apache 2 license](./LICENSE).
\ No newline at end of file
......@@ -12,9 +12,188 @@
// See the License for the specific language governing permissions and
// limitations under the License.
// tag::description[]
//! Implements a serialization and deserialization codec for simple marshalling.
// end::description[]
//! # Parity SCALE Codec
//!
//! Rust implementation of the SCALE (Simple Concatenated Aggregate Little-Endian) data format
//! for types used in the Parity Substrate framework.
//!
//! SCALE is a light-weight format which allows encoding (and decoding) which makes it highly
//! suitable for resource-constrained execution environments like blockchain runtimes and low-power,
//! low-memory devices.
//!
//! It is important to note that the encoding context (knowledge of how the types and data structures look)
//! needs to be known separately at both encoding and decoding ends.
//! The encoded data does not include this contextual information.
//!
//! To get a better understanding of how the encoding is done for different types,
//! take a look at the
//! [low-level data formats overview page at the Substrate docs site](https://docs.substrate.dev/docs/low-level-data-formats).
//!
//! ## Implementation
//!
//! The codec is implemented using the following traits:
//!
//! ### Encode
//!
//! The `Encode` trait is used for encoding of data into the SCALE format. The `Encode` trait contains the following functions:
//! * `size_hint(&self) -> usize`: Gets the capacity (in bytes) required for the encoded data.
//! This is to avoid double-allocation of memory needed for the encoding.
//! It can be an estimate and does not need to be an exact number.
//! If the size is not known, even no good maximum, then we can skip this function from the trait implementation.
//! This is required to be a cheap operation, so should not involve iterations etc.
//! * `encode_to<T: Output>(&self, dest: &mut T)`: Encodes the value and appends it to a destination buffer.
//! * `encode(&self) -> Vec<u8>`: Encodes the type data and returns a slice.
//! * `using_encoded<R, F: FnOnce(&[u8]) -> R>(&self, f: F) -> R`: Encodes the type data and executes a closure on the encoded value.
//! Returns the result from the executed closure.
//!
//! **Note:** Implementations should override `using_encoded` for value types and `encode_to` for allocating types.
//! `size_hint` should be implemented for all types, wherever possible. Wrapper types should override all methods.
//!
//! ### Decode
//!
//! The `Decode` trait is used for deserialization/decoding of encoded data into the respective types.
//!
//! * `fn decode<I: Input>(value: &mut I) -> Result<Self, Error>`: Tries to decode the value from SCALE format to the type it is called on.
//! Returns an `Err` if the decoding fails.
//!
//! ### CompactAs
//!
//! The `CompactAs` trait is used for wrapping custom types/structs as compact types, which makes them even more space/memory efficient.
//! The compact encoding is described [here](https://docs.substrate.dev/docs/low-level-data-formats#section-compactgeneral-integers).
//!
//! * `encode_as(&self) -> &Self::As`: Encodes the type (self) as a compact type.
//! The type `As` is defined in the same trait and its implementation should be compact encode-able.
//! * `decode_from(_: Self::As) -> Self`: Decodes the type (self) from a compact encode-able type.
//!
//! ### HasCompact
//!
//! The `HasCompact` trait, if implemented, tells that the corresponding type is a compact encode-able type.
//!
//! ## Usage Examples
//!
//! Following are some examples to demonstrate usage of the codec.
//!
//! ### Simple types
//!
//! ```
//! use parity_scale_codec_derive::{Encode, Decode};
//! use parity_scale_codec::{Encode, Decode};
//!
//! #[derive(Debug, PartialEq, Encode, Decode)]
//! enum EnumType {
//! #[codec(index = "15")]
//! A,
//! B(u32, u64),
//! C {
//! a: u32,
//! b: u64,
//! },
//! }
//!
//! let a = EnumType::A;
//! let b = EnumType::B(1, 2);
//! let c = EnumType::C { a: 1, b: 2 };
//!
//! a.using_encoded(|ref slice| {
//! assert_eq!(slice, &b"\x0f");
//! });
//!
//! b.using_encoded(|ref slice| {
//! assert_eq!(slice, &b"\x01\x01\0\0\0\x02\0\0\0\0\0\0\0");
//! });
//!
//! c.using_encoded(|ref slice| {
//! assert_eq!(slice, &b"\x02\x01\0\0\0\x02\0\0\0\0\0\0\0");
//! });
//!
//! let mut da: &[u8] = b"\x0f";
//! assert_eq!(EnumType::decode(&mut da).ok(), Some(a));
//!
//! let mut db: &[u8] = b"\x01\x01\0\0\0\x02\0\0\0\0\0\0\0";
//! assert_eq!(EnumType::decode(&mut db).ok(), Some(b));
//!
//! let mut dc: &[u8] = b"\x02\x01\0\0\0\x02\0\0\0\0\0\0\0";
//! assert_eq!(EnumType::decode(&mut dc).ok(), Some(c));
//!
//! let mut dz: &[u8] = &[0];
//! assert_eq!(EnumType::decode(&mut dz).ok(), None);
//!
//! # fn main() { }
//! ```
//!
//! ### Compact type with HasCompact
//!
//! ```
//! use parity_scale_codec_derive::{Encode, Decode};;
//! use parity_scale_codec::{Encode, Decode, Compact, HasCompact};
//!
//! #[derive(Debug, PartialEq, Encode, Decode)]
//! struct Test1CompactHasCompact<T: HasCompact> {
//! #[codec(compact)]
//! bar: T,
//! }
//!
//! #[derive(Debug, PartialEq, Encode, Decode)]
//! struct Test1HasCompact<T: HasCompact> {
//! #[codec(encoded_as = "<T as HasCompact>::Type")]
//! bar: T,
//! }
//!
//! let test_val: (u64, usize) = (0u64, 1usize);
//!
//! let encoded = Test1HasCompact { bar: test_val.0 }.encode();
//! assert_eq!(encoded.len(), test_val.1);
//! assert_eq!(<Test1CompactHasCompact<u64>>::decode(&mut &encoded[..]).unwrap().bar, test_val.0);
//!
//! # fn main() { }
//! ```
//! ### Type with CompactAs
//!
//! ```rust
//!
//! use serde_derive::{Serialize, Deserialize};
//! use parity_scale_codec_derive::{Encode, Decode};;
//! use parity_scale_codec::{Encode, Decode, Compact, HasCompact, CompactAs};
//!
//! #[cfg_attr(feature = "std", derive(Serialize, Deserialize, Debug))]
//! #[derive(PartialEq, Eq, Clone)]
//! struct StructHasCompact(u32);
//!
//! impl CompactAs for StructHasCompact {
//! type As = u32;
//!
//! fn encode_as(&self) -> &Self::As {
//! &12
//! }
//!
//! fn decode_from(_: Self::As) -> Self {
//! StructHasCompact(12)
//! }
//! }
//!
//! impl From<Compact<StructHasCompact>> for StructHasCompact {
//! fn from(_: Compact<StructHasCompact>) -> Self {
//! StructHasCompact(12)
//! }
//! }
//!
//! #[derive(Debug, PartialEq, Encode, Decode)]
//! enum TestGenericHasCompact<T> {
//! A {
//! #[codec(compact)] a: T
//! },
//! }
//!
//! let a = TestGenericHasCompact::A::<StructHasCompact> {
//! a: StructHasCompact(12325678),
//! };
//!
//! let encoded = a.encode();
//! assert_eq!(encoded.len(), 2);
//!
//! # fn main() { }
//! ```
#![warn(missing_docs)]
......
......@@ -372,7 +372,7 @@ fn generic_bound_hascompact() {
#[cfg_attr(feature = "std", derive(Serialize, Deserialize, Debug))]
#[derive(PartialEq, Eq, Clone)]
// This struct does not impl Codec
struct StructHasCompact(u8);
struct StructHasCompact(u32);
impl CompactAs for StructHasCompact {
type As = u32;
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment