Substrate is a blockchain platform with a completely generic state transition function. That said, it does come with both standards and conventions (particularly regarding the Runtime Module Library) regarding underlying data structures. Roughly speaking, these core data types correspond to +trait+s in terms of the actual non-negotiable standard and generic +struct+s in terms of the convention.

Extrinsics in Substrate are pieces of information from "the outside world" that are contained in the blocks of the chain. You might think "ahh, that means transactions": in fact, no. Extrinsics fall into two broad categories of which only one is transactions. The other is known as inherents. The difference between these two is that transactions are signed and gossiped on the network and can be deemed useful per se. This fits the mold of what you would call transactions in Bitcoin or Ethereum.

Inherents, meanwhile, are not passed on the network and are not signed. They represent data which describes the environment but which cannot call upon anything to prove it such as a signature. Rather they are assumed to be "true" simply because a sufficiently large number of validators have agreed on them being reasonable.

To give an example, there is the timestamp inherent, which sets the current timestamp of the block. This is not a fixed part of Substrate, but does come as part of FRAME to be used as desired. No signature could fundamentally prove that a block were authored at a given time in quite the same way that a signature can "prove" the desire to spend some particular funds. Rather, it is the business of each validator to ensure that they believe the timestamp is set to something reasonable before they agree that the block candidate is valid.

Other examples include the parachain-heads extrinsic in Polkadot and the "note-missed-proposal" extrinsic used in FRAME to determine and punish or deactivate offline validators.

Substrate chains all have a runtime. The runtime is a WebAssembly "blob" that includes a number of entry-points. Some entry-points are required as part of the underlying Substrate specification. Others are merely convention and required for the default implementation of the Substrate client to be able to author blocks.

If you want to develop a chain with Substrate, you will need to implement the Core trait. This Core trait generates an API with the minimum necessary functionality to interact with your runtime. A special macro is provided called impl_runtime_apis! that help you implement runtime API traits. All runtime API trait implementations need to be done in one call of the impl_runtime_apis! macro. All parameters and return values need to implement parity-codec to be encodable and decodable.

Here’s a snippet of the Polkadot API implementation as of PoC-3:

Substrate FRAME includes functionality for timestamps and slashing. If used, these rely on "trusted" external information being passed in via inherent extrinsics. The Substrate reference block authoring client software will expect to be able to call into the runtime API with collated data (in the case of the reference Substrate authoring client, this is merely the current timestamp and which nodes were offline) in order to return the appropriate extrinsics ready for inclusion. If new inherent extrinsic types and data are to be used in a modified runtime, then it is this function (and its argument type) that would change.

In Substrate, there is a major distinction between blockchain syncing and block authoring ("authoring" is a more general term for what is called "mining" in Bitcoin). The first case might be referred to as a "full node" (or "light node" - Substrate supports both): authoring necessarily requires a synced node and, therefore, all authoring clients must necessarily be able to synchronize. However, the reverse is not true. The primary functionality that authoring nodes have which is not in "sync nodes" is threefold: transaction queue logic, inherent transaction knowledge and BFT consensus logic. BFT consensus logic is provided as a core element of Substrate and can be ignored since it is only exposed in the SDK under the authorities() API entry.

Transaction queue logic in Substrate is designed to be as generic as possible, allowing a runtime to express which transactions are fit for inclusion in a block through the initialize_block and apply_extrinsic calls. However, more subtle aspects like prioritization and replacement policy must currently be expressed "hard coded" as part of the blockchain’s authoring code. That said, Substrate’s reference implementation for a transaction queue should be sufficient for an initial chain implementation.

Inherent extrinsic knowledge is again somewhat generic, and the actual construction of the extrinsics is, by convention, delegated to the "soft code" in the runtime. If ever there needs to be additional extrinsic information in the chain, then both the block authoring logic will need to be altered to provide it into the runtime and the runtime’s inherent_extrinsics call will need to use this extra information in order to construct any additional extrinsic transactions for inclusion in the block.

• 0.1 "PoC-1": PBFT consensus, Wasm runtime engine, basic runtime modules.

• 0.2 "PoC-2": Libp2p

• AfG consensus

• Improved PoS

• Smart contract runtime module

• Splitting out runtime modules into separate repo

• Introduce substrate executable (the skeleton-key runtime)

• Introduce basic but extensible transaction queue and block-builder and place them in the executable.

• DAO runtime module

• Audit

To get going as fast as possible, there is a simple script that installs all required dependencies and installs Substrate into your path. Just open a terminal and run:

You can start a local Substrate development chain with running substrate --dev.

To create your own global network/cryptocurrency, you’ll need to make a new Substrate Node chain specification file ("chainspec").

First let’s get a template chainspec that you can edit. We’ll use the "staging" chain, a sort of default chain that the node comes pre-configured with:

Now, edit ~/chainspec.json in your editor. There are a lot of individual fields for each module, and one very large one which contains the WebAssembly code blob for this chain. The easiest field to edit is the block period. Change it to 10 (seconds):

Now with this new chainspec file, you can build a "raw" chain definition for your new chain:

This can be fed into Substrate:

It won’t do much until you start producing blocks though, so to do that you’ll need to use the --validator option together with passing the seed for the account(s) that is configured to be the initial authorities:

You can distribute mychain.json so that everyone can synchronize and (depending on your authorities list) validate on your chain.

If you’d actually like to hack on Substrate, you can just grab the source code and build it. Ensure you have Rust and the support software installed:

6.1.1. Linux and Mac

curl https://sh.rustup.rs -sSf | sh

rustup update nightly
rustup target add wasm32-unknown-unknown --toolchain nightly
rustup update stable

6.1.2. Windows

6.1.3. Docker

#run it in the folder with the Substrate source code
docker run --rm -it -w /shellhere/substrate \
                    -v $(pwd):/shellhere/substrate \
                    paritytech/ci-linux:production <cargo commands>

6.1.4. Shared Steps

git clone https://github.com/paritytech/substrate.git
cd substrate

cargo build                 # Builds all native code

cargo test --all

cargo run --release -- --dev

cargo run --release \-- \
  --base-path /tmp/alice \
  --chain=local \
  --alice \
  --node-key 0000000000000000000000000000000000000000000000000000000000000001 \
  --telemetry-url 'ws://telemetry.polkadot.io:1024 0' \
  --validator

cargo run --release \-- \
  --base-path /tmp/bob \
  --bootnodes /ip4/127.0.0.1/tcp/30333/p2p/QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR \
  --chain=local \
  --bob \
  --port 30334 \
  --telemetry-url 'ws://telemetry.polkadot.io:1024 0' \
  --validator

Flaming Fir is the new testnet for Substrate master (2.0) to test the latest development features. Please note that master is not compatible with the BBQ Birch, Charred Cherry, Dried Danta or Emberic Elm testnets. Ensure you have the dependencies listed above before compiling.

Since Flaming Fir is targeting the master branch we make absolutely no guarantees of stability and/or persistence of the network. We might reset the chain at any time if it is necessary to deploy new changes. Currently, the validators are running with a client built from d013bd900, if you build from this commit you should be able to successfully sync, later commits may not work as new breaking changes may be introduced in master.

Latest known working version: a2a0eb5398d6223e531455b4c155ef053a4a3a2b

You can run the tests if you like:

Start your node:

To see a list of command line options, enter:

For example, you can choose a custom node name:

If you are successful, you will see your node syncing at https://telemetry.polkadot.io/#/Flaming%20Fir

Emberic Elm is the testnet for Substrate 1.0. Please note that 1.0 is not compatible with the BBQ Birch, Charred Cherry, Dried Danta or Flaming Fir testnets. In order to join the Emberic Elm testnet you should build from the v1.0 branch. Ensure you have the dependencies listed above before compiling.

You can then follow the same steps for building and running as described above in Joining the Flaming Fir Testnet.

For most users who want to run a validator node, the author_rotateKeys RPC call is sufficient. The RPC call will generate N Session keys for you and return their public keys. N is the number of session keys configured in the runtime. The output of the RPC call can be used as input for the session::set_keys transaction.

If the Session keys need to match a fixed seed, they can be set individually key by key. The RPC call expects the key seed and the key type. The key types supported by default in Substrate are listed here, but the user can declare any key type.

KEY_TYPE - needs to be replaced with the 4-character key type identifier. SEED - is the seed of the key. PUBLIC - public key for the given key.

You can generate documentation for a Substrate Rust package and have it automatically open in your web browser using rustdoc with Cargo, (of the The Rustdoc Book), by running the following command:

Replacing <spec> with one of the following (i.e. cargo doc --package substrate --open):

• All Substrate Packages

  substrate

• Substrate Core

  substrate, substrate-cli, substrate-client, substrate-client-db,
  substrate-consensus-common, substrate-consensus-rhd,
  substrate-executor, substrate-finality-grandpa, substrate-keyring, substrate-keystore, substrate-network,
  substrate-network-libp2p, substrate-primitives, substrate-rpc, substrate-rpc-servers,
  substrate-serializer, substrate-service, substrate-service-test, substrate-state-db,
  substrate-state-machine, substrate-telemetry, substrate-test-client,
  substrate-test-runtime, substrate-transaction-graph, sp-transaction-pool,
  substrate-trie

• Substrate Runtime

  sr-api, sr-io, sr-primitives, sr-sandbox, sr-std, sr-version

• FRAME Core

  frame-metadata, frame-support, frame-system

• FRAME Pallets

  pallet-assets, pallet-balances, pallet-consensus, pallet-contracts, pallet-council, pallet-democracy, pallet-example,
  frame-executive,  pallet-session, pallet-staking, pallet-timestamp, pallet-treasury

• Node

  node-cli, node-consensus, node-executor, node-network, node-primitives, node-runtime

• Subkey

  subkey

Document source code for Substrate packages by annotating the source code with documentation comments.

Example (generic):

• Important notes:

  • Documentation comments must use annotations with a triple slash ///

  • Modules are documented using //!

• Special section header is indicated with a hash #.

  • Panics section requires an explanation if the function triggers a panic

  • Errors section is for describing conditions under which a function of method returns Err(E) if it returns a Result<T, E>

  • Safety section requires an explanation if the function is unsafe

  • Examples section includes examples of using the function or method

• Code block annotations for examples are included between triple graves, as shown above. Instead of including the programming language to use for syntax highlighting as the annotation after the triple graves, alternative annotations include the ignore, text, should_panic, or no_run.

• Summary sentence is a short high level single sentence of its functionality

• Description paragraph is for details additional to the summary sentence

• Missing documentation annotations may be used to identify where to generate warnings with ![warn(missing_docs)] or errors ![deny(missing_docs)]

• Hide documentation for items with #[doc(hidden)]

The code block annotations in the # Example section may be used as documentation as tests and for extended examples.

• Important notes:

  • Rustdoc will automatically add a main() wrapper around the code block to test it

  • Documenting macros.

  • Documentation as tests examples are included when running cargo test

All pull requests will be checked against either Polkadot master, or your provided Polkadot companion PR. That is, If your PR changes the external APIs or interfaces used by Polkadot. If you tagged the PR with breaksapi or breaksconsensus this is most certainly the case, in all other cases check for it by running step 1 below.

To create a Polkadot companion PR:

1. Pull latest Polkadot master (or clone it, if you haven’t yet).

2. Override substrate deps to point to your local path or branch using https://github.com/bkchr/diener. (E.g. from the polkadot clone dir run diener patch --crates-to-patch ../substrate --substrate assuming substrate clone is in a sibling dir. If you do use diener, ensure that you do not commit the changes diener makes to the Cargo.tomls.)

3. Make the changes required and build polkadot locally.

4. Submit all this as a PR against the Polkadot Repo.

5. In the description of your Substrate PR add "polkadot companion: [Polkadot_PR_URL]"

6. Now you should see that the check_polkadot CI job will build your Substrate PR agains the mentioned Polkadot branch in your PR description.

7. Someone will need to approve the Polkadot PR before the Substrate CI will go green. (The Polkadot CI failing can be ignored as long as the polkadot job in the substrate PR is green).

8. Wait for reviews on both the Substrate and the Polkadot PRs.

9. Once the Substrate PR runs green, a member of the parity github group can comment on the Substrate PR with bot merge which will:

   • Merge the Substrate PR.

   • The bot will push a commit to the Polkadot PR updating its Substrate reference. (effecively doing cargo update -p sp-io)

   • If the polkadot PR origins from a fork then a project member may need to press approve run on the polkadot PR.

   • The bot will merge the Polkadot PR once all its CI {"build_allow_failure":false} checks are green. Note: The merge-bot currently doesn’t work with forks on org accounts, only individual accounts. (Hint: it’s recommended to use bot merge to merge all substrate PRs, not just ones with a polkadot companion.)

If your PR is reviewed well, but a Polkadot PR is missing, signal it with A7-needspolkadotpr to prevent it from getting automatically merged.

As there might be multiple pending PRs that might conflict with one another, a) you should not merge the substrate PR until the Polkadot PR has also been reviewed and b) both should be merged pretty quickly after another to not block others.

We recognise that this software is merely a tool for users to create and maintain their blockchain of preference. We see that blockchains are naturally community platforms with users being the ultimate decision makers. We assert that good software will maximise user agency by facilitate user-expression on the network. As such:

• This project will strive to give users as much choice as is both reasonable and possible over what protocol they adhere to; but

• use of the project’s technical forums, commenting systems, pull requests and issue trackers as a means to express individual protocol preferences is forbidden.