Changes (partial https://github.com/paritytech/polkadot-sdk/issues/994):
- Set log to `0.4.20` everywhere
- Lift `log` to the workspace

Starting with a simpler one after seeing
https://github.com/paritytech/polkadot-sdk/pull/2065 from @jsdw

.
This sets the `default-features` to `false` in the root and then
overwrites that in each create to its original value. This is necessary
since otherwise the `default` features are additive and its impossible
to disable them in the crate again once they are enabled in the
workspace.

I am using a tool to do this, so its mostly a test to see that it works
as expected.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

This PR adds initial support for building RISC-V runtimes targeting
PolkaVM.

- Setting the `SUBSTRATE_RUNTIME_TARGET=riscv` environment variable will
now build a RISC-V runtime instead of a WASM runtime.
- This only adds support for *building* runtimes; running them will need
a PolkaVM-based executor, which I will add in a future PR.
- Only building the minimal runtime is supported (building the Polkadot
runtime doesn't work *yet* due to one of the dependencies).
- The builder now sets a `substrate_runtime` cfg flag when building the
runtimes, with the idea being that instead of doing `#[cfg(not(feature =
"std"))]` or `#[cfg(target_arch = "wasm32")]` to detect that we're
building a runtime you'll do `#[cfg(substrate_runtime)]`. (Switching the
whole codebase to use this will be done in a future PR; I deliberately
didn't do this here to keep this PR minimal and reviewable.)
- Further renaming of things (e.g. types, environment variables and proc
macro attributes having "wasm" in their name) to be target-agnostic will
also be done in a future refactoring PR (while keeping backwards
compatibility where it makes sense; I don't intend to break anyone's
workflow or create unnecessary churn).
- This PR also fixes two bugs in the `wasm-builder` crate:
* The `RUSTC` environment variable is now removed when invoking the
compiler. This prevents the toolchain version from being overridden when
called from a `build.rs` script.
* When parsing the `rustup toolchain list` output the `(default)` is now
properly stripped and not treated as part of the version.
- I've also added a minimal CI job that makes sure this doesn't break in
the future. (cc @paritytech/ci)

cc @athei



------

Also, just a fun little tidbit: quickly comparing the size of the built
runtimes it seems that the PolkaVM runtime is slightly smaller than the
WASM one. (`production` build, with the `names` section substracted from
the WASM's size to keep things fair, since for the PolkaVM runtime we're
currently stripping out everything)

- `.wasm`: 625505 bytes
- `.wasm` (after wasm-opt -O3): 563205 bytes
- `.wasm` (after wasm-opt -Os): 562987 bytes
- `.wasm` (after wasm-opt -Oz): 536852 bytes
- `.polkavm`: ~~580338 bytes~~ 550476 bytes (after enabling extra target
features; I'll add those in another PR once we have an executor working)

---------

Co-authored-by: Bastian Köcher <[email protected]>

Related https://github.com/paritytech/polkadot-sdk/issues/3032

---

Using https://github.com/liamaharon/cargo-workspace-version-tools/ 

`cargo run -- sync --path ../polkadot-sdk`

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

Step towards https://github.com/paritytech/polkadot-sdk/issues/1975

As reported
https://github.com/paritytech/polkadot-sdk/issues/1975#issuecomment-1774534225
I'd like to encapsulate crypto related stuff in a dedicated folder.

Currently all cryptographic primitive wrappers are all sparsed in
`substrate/core` which contains "misc core" stuff.

To simplify the process, as the first step with this PR I propose to
move the cryptographic hashing there.

The `substrate/crypto` folder was already created to contains `ec-utils`
crate.

Notes:
- rename `sp-core-hashing` to `sp-crypto-hashing`
- rename `sp-core-hashing-proc-macro` to `sp-crypto-hashing-proc-macro`
- As the crates name is changed I took the freedom to restart fresh from
version 0.1.0 for both crates

---------

Co-authored-by: Robert Hambrock <[email protected]>

We currently use a bit of a hack in `.cargo/config` to make sure that
clippy isn't too annoying by specifying the list of lints.

There is now a stable way to define lints for a workspace. The only down
side is that every crate seems to have to opt into this so there's a
*few* files modified in this PR.

Dependencies:

- [x] PR that upgrades CI to use rust 1.74 is merged.

---------

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

Bumps [tracing-core](https://github.com/tokio-rs/tracing) from 0.1.31 to
0.1.32.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/tracing/releases">tracing-core's
releases</a>.</em></p>
<blockquote>
<h2>tracing-core 0.1.32</h2>
<h3>Documented</h3>
<ul>
<li>Fix typo in <code>field</code> docs (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2611">#2611</a>)</li>
<li>Remove duplicate wording (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2674">#2674</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li>Allow <code>ValueSet</code>s of any length (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2508">#2508</a>)</li>
</ul>
<p><a
href="https://redirect.github.com/tokio-rs/tracing/issues/2611">#2611</a>:
<a
href="https://redirect.github.com/tokio-rs/tracing/pull/2611">tokio-rs/tracing#2611</a>
<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2674">#2674</a>:
<a
href="https://redirect.github.com/tokio-rs/tracing/pull/2674">tokio-rs/tracing#2674</a>
<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2508">#2508</a>:
<a
href="https://redirect.github.com/tokio-rs/tracing/pull/2508">tokio-rs/tracing#2508</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/tokio-rs/tracing/commit/c4b2a56937dd40aaa2e2991636eca6748353201f"><code>c4b2a56</code></a>
chore: prepare tracing-core 0.1.32 (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2754">#2754</a>)</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/2502f19d934b092fc01bb7493eacbb16b4038bf3"><code>2502f19</code></a>
chore: prepare tracing-attributes 0.1.27 (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2756">#2756</a>)</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/90487620d8fd4b0a44e6a0385bda3643bf6f19a2"><code>9048762</code></a>
Revert &quot;log: update to env_logger 0.10 to fix GHSA-g98v-hv3f-hcfr
(<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2740">#2740</a>)&quot;
(#...</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/6ba5af2ce2a814fea521d2d0a5ea1e88d7b6011e"><code>6ba5af2</code></a>
docs: remove mention of <code>Registration</code> on v0.1.x (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2753">#2753</a>)</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/11aac9a07c4aac35b12ea82d8a5b1cb00a118928"><code>11aac9a</code></a>
log: deprecate <code>env_logger</code> in favor of
<code>tracing_subscriber::fmt::Subscriber</code>...</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/2f27752a9912f03ed64e2b29e1ea06ebb4b09e83"><code>2f27752</code></a>
chore: remove <code>env_logger</code> from <code>hyper</code>
example</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/f96846d78a3f240a2c81636a7d6921e5d74ac79f"><code>f96846d</code></a>
attributes: fix typo &quot;overriden&quot; =&gt; &quot;overridden&quot;
(<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2719">#2719</a>)</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/71b5b2c5792ee6cd4e98112a90f9fcd19dd1ddc0"><code>71b5b2c</code></a>
subscriber: make <code>format::Writer::new()</code> public (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2680">#2680</a>)</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/f8c000202a30f8b9814b21a1e949ab35aeca5954"><code>f8c0002</code></a>
attributes: fix clippy warning in attributes tests (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2742">#2742</a>)</li>
<li><a
href="https://github.com/tokio-rs/tracing/commit/60b2dc34662648b91d69b2fa51d18d3677ddbce7"><code>60b2dc3</code></a>
journald: fix clippy <code>unwrap_or_default</code> warning (<a
href="https://redirect.github.com/tokio-rs/tracing/issues/2742">#2742</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/tokio-rs/tracing/compare/tracing-core-0.1.31...tracing-core-0.1.32">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=tracing-core&package-manager=cargo&previous-version=0.1.31&new-version=0.1.32)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Liam Aharon <[email protected]>

Using taplo, fixes all our broken and inconsistent toml formatting and
adds CI to keep them tidy.

If people want we can customise the format rules as described here
https://taplo.tamasfe.dev/configuration/formatter-options.html

@ggwpez

, I suggest zepter is used only for checking features are
propagated, and leave formatting for taplo to avoid duplicate work and
conflicts.

TODO
- [x] Use `exclude = [...]` syntax in taplo file to ignore zombienet
tests instead of deleting the dir

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Bumps [secp256k1](https://github.com/rust-bitcoin/rust-secp256k1) from
0.24.3 to 0.28.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-bitcoin/rust-secp256k1/blob/master/CHANGELOG.md">secp256k1's
changelog</a>.</em></p>
<blockquote>
<h1>0.28.0 - 2023-10-23</h1>
<ul>
<li>Add bindings to the ElligatorSwift implementation <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/627">#627</a></li>
<li>Depend on recent release of <code>bitcoin_hashes</code> v0.13.0 <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/621">#621</a></li>
<li>Add a verify function to <code>PublicKey</code> <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/618">#618</a></li>
<li>Add serialize function for schnorr::Signature <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/607">#607</a></li>
<li>Bump MSRV to 1.48 <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/595">#595</a></li>
<li>Remove implementations of <code>PartialEq</code>, <code>Eq</code>,
<code>PartialOrd</code>, <code>Ord</code>, and <code>Hash</code> from
the
<code>impl_array_newtype</code> macro. Users will now need to derive
these traits if they are wanted.</li>
</ul>
<h1>0.27.0 - 2023-03-15</h1>
<ul>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/588">Depend
on newly release <code>bitcoin_hashes</code> v0.12</a>.</li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/578">Implement
<code>Debug</code> trait for <code>Scalar</code> type</a>.</li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/582">Implement
<code>insecure-erase</code></a>.</li>
</ul>
<h1>0.26.0 - 2202-12-19</h1>
<ul>
<li>Update libsecp25k1 to v0.2.0</li>
</ul>
<h1>0.25.0 - 2022-12-07</h1>
<ul>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/548">Fix
soundness issue with <code>preallocated_gen_new</code></a></li>
<li>Update to <code>secp256k1-sys</code> <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/549">v0.7.0</a></li>
<li>Use type system to <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/483">improve
safety</a>.</li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/490">Change
secp256k1-sys symbol names to 0_6_1</a>.</li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/499">Introduce
<code>rustfmt</code></a> to the codebase.</li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/507">Make
all raw pointer methods go through the CPtr trait</a>.</li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/518">Make
comparison functions stable</a>.</li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/512">Remove</a>
public constant <code>ONE_KEY</code> (consider using
<code>FromStr</code> as a replacement).</li>
</ul>
<h1>0.24.1 - 2022-10-25</h1>
<ul>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/issues/491">Fix
broken deserialization logic of <code>KeyPair</code></a> that previously
always panicked. After the patch deserialization only panics if neither
the <code>global-context</code> nor the <code>alloc</code> (default)
feature is active.</li>
</ul>
<h1>0.24.0 - 2022-07-20</h1>
<ul>
<li>Upgrade to new release of <a
href="https://github.com/rust-bitcoin/bitcoin_hashes/releases/tag/0.11.0">bitcoin_hashes</a>.</li>
</ul>
<h1>0.23.4 - 2022-07-14</h1>
<ul>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/474">Disable
automatic rerandomization of contexts under WASM</a></li>
</ul>
<h1>0.23.3 - 2022-06-29</h1>
<ul>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/465">Add
must_use for mut self key manipulation methods</a></li>
<li><a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/pull/466">Fix
fuzzing feature guard</a></li>
</ul>
<h1>0.23.2 - 2022-06-27</h1>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/7de09c8050da12a13ef9ee3850597f69c887952d"><code>7de09c8</code></a>
Merge <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/issues/256">rust-bitcoin/rust-secp256k1256</a>:
Tracking PR for release: `secp256k1 v0...</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/3dc5b165401f249c01a88cec54061301cffd97a0"><code>3dc5b16</code></a>
Bump version to v0.28.0</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/3aada83180beec2b9f5ab8e7b9280a5517d3bcde"><code>3aada83</code></a>
Merge <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/issues/256">rust-bitcoin/rust-secp256k1256</a>:
Add bindings to the ElligatorSwift imp...</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/39febcb866ce285d53052a3636602f04483aa710"><code>39febcb</code></a>
Create rust-bidings</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/da4f67b274c3061717723a568cfb182e4e2e7cce"><code>da4f67b</code></a>
Merge <a
href="https://redirect.github.com/rust-bitcoin/rust-secp256k1/issues/256">rust-bitcoin/rust-secp256k1256</a>:
Update vendored lib secp256k1 to v0.4.0</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/80b2a8d4aa6ffa72041d569eab2278cd8c1ace2a"><code>80b2a8d</code></a>
Update vendored libsecp to v0.4.0</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/d2285c929a086276ce6d1670d795c49191e30c65"><code>d2285c9</code></a>
ci: Remove MIPS* from CI</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/0d58f50d523b40a78de0b87146208e3ad338c8ba"><code>0d58f50</code></a>
ci: generalize grp in &quot;illegal callback&quot; test</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/acf9ac13e9f8df84dd52d2f012cda7211a6af10c"><code>acf9ac1</code></a>
delete <code>test_manual_create_destroy</code> test</li>
<li><a
href="https://github.com/rust-bitcoin/rust-secp256k1/commit/04ce50891bb0d49be5355f5c0d82db70d7dda65a"><code>04ce508</code></a>
lib: fix bad unit test</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-bitcoin/rust-secp256k1/compare/secp256k1-0.24.3...secp256k1-0.28.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=secp256k1&package-manager=cargo&previous-version=0.24.3&new-version=0.28.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Liam Aharon <[email protected]>

Next step in process of making BEEFY being able to generate both ECDSA
and BLS signature after #1705. It allows BEEFY to use a pair of ECDSA
and BLS key as a AuthorityId.

---------

Co-authored-by: Davide Galassi <[email protected]>
Co-authored-by: Robert Hambrock <[email protected]>

Adds a config file that allows to run `zepter` without any arguments in
the workspace to address all issues.
A secondary workflow for the CI is provided as `zepter run check`. Both
the formatting and linting are now in one check for efficiancy.

The latest version also detects some more things that `featalign` was
already showing.

Error message [in the
CI](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3916205)
now looks like this:
```pre
...
crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml)
  feature 'std'
    must propagate to:
      parity-scale-codec
Found 55 issues (run with --fix to fix).
Error: Command 'lint propagate-feature' failed with exit code 1

Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration.
It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter`.
Otherwise please ask directly in the Merge Request, GitHub Discussions or on Matrix Chat, thank you.

For more information, see:
  - https://github.com/paritytech/polkadot-sdk/issues/1831
  - https://github.com/ggwpez/zepter
```

TODO:
- [x] Check that CI fails correctly

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update to ed25519-dalek v2
* Update Cargo.lock
* Remove default features
* Update cumulus/bridges/primitives/test-utils/src/keyring.rs

Co-authored-by: Bastian Köcher <[email protected]>

---------

Co-authored-by: Bastian Köcher <[email protected]>

* Add markdown linting

- add linter default rules
- adapt rules to current code
- fix the code for linting to pass
- add CI check

fix #1243

* Fix markdown for Substrate
* Fix tooling install
* Fix workflow
* Add documentation
* Remove trailing spaces
* Update .github/.markdownlint.yaml

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
* Fix mangled markdown/lists
* Fix captalization issues on known words

* Fix build profiles

Closes https://github.com/paritytech/polkadot-sdk/issues/1155

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Manually set version to 1.0.0

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use workspace repo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* 'Authors and Edition from workspace

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: alvicsam <[email protected]>

* CI: Add feature sorting check

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Sort all features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add some mistakes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Add some mistakes"

This reverts commit b2b1099f979f6decb22d09b46689c1554bb72e81.

* CI job naming

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add oneliner formatting

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Explain tool

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use latest version

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Better erorr message

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Format after master merge

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use --check option

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Messed up the merge commit...

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* bump ed25519-dalek to 2.0

* sp-statement-store: don't use deprecated SecretKey::new

* Remove snow patch

---------

Co-authored-by: Sebastian Kunert <[email protected]>

* Introduce bandersnatch vrf

* Some documentation

* Fix tests

* Fix docs refs

* Some more docs

* Comments about key derivation

* Make clippy happy

* Fix ring context enc/dec test

* Fix docs

* Switch to upstream ring-vrf

* Use sub-domains to construct VrfInput

* Bandersnatch VRF experimental feature

* Restore upstream dep

* Fix feature flags

* Apply typo fix

Co-authored-by: Anton <[email protected]>

* Bump bandersnatch-vrfs

* Weiestrass form has been selected

* Rename bandersnatch testing app crypto id

* Support for seed recovery

* Clarified domain size <-> key size relationship

* cargo fmt

* Trigger CI

* Some required tweaks to crypto types

* Remove leftovers from Cargo.toml

* Remove some TODO notes

* Simplification of structs construction

* Trigger CI

* Apply review suggestion

Co-authored-by: Koute <[email protected]>

* Docs typo

* Fix keystore tests

* Consistence

* Add ref to git rependency

* Static check of MAX_VRF_IOS value

* Clarify behavior for out of ring keys signatures

* Add test for ring-vrf to the keystore

* Fix docs

---------

Co-authored-by: Anton <[email protected]>
Co-authored-by: Koute <[email protected]>

* Merged BEEFY primitives with generic signature and keyset commitment support from old pull to current code

* - Add bls-experimental feature to application-crypto and beefy primitives
- Fix remaining crypto -> ecdsa_crypto
- code build but not tests

* Make beefy primitive tests compile

* move bls related beefy primitives code and test behind bls-experimental flag

* Make BEEFY clients complies with BEEFY API depending on AuthorityId

* - Rename `BeefyAuthoritySet.root` → `BeefyAuthoritySet.keyset_commitment`.
- Remove apk proof keyset_commitment from `BeefyAuthoritySet`.
- Fix failing signed commitment and signature to witness test.
- Make client compatible with BeefyAPI generic on AuthorityId.
- `crypto` → `ecdsa_crypto` in BEEFY client and frame.

* Commit Cargo lock remove ark-serialize from BEEFY primitives

* Use Codec instead of Encode + Decode in primitives/consensus/beefy/src/lib.rs

Co-authored-by: Davide Galassi <[email protected]>

* - Make `BeefyApi` generic over Signature type.
- Make new `BeeyApi` functinos also generic over AuthorityId and Signature

* Unmake BeefyAPI generic over Signature. Recover Signature type from AuthId.

* - dont use hex or hex-literal use array-bytes instead in beefy primitives and bls crypto.
- CamelCase ECDSA and BLS everywhere.

* Move the definition of BEEFY key type from `primitives/beefy` to `crypto.rs` according to new convention.

* - Add bls377_generate_new to `sp-io` and `application_crypto::bls`.
- Add `bls-experimental` to `sp-io`

Does not compile because PassByCodec can not derive PassBy using customly implemented PassByIner.

* Implement PassBy for `bls::Public` manually

* fix Beefy `KEY_TYPE` in `frame/beefy` tests to come from `sp-core::key_types` enum

* specify both generic for `hex2array_unchecked` in `sp-core/bls.rs`

* Rename `crypto`→`ecdsa_crypto` in `primitives/consensus/beefy/src/test_utils.rs` docs

* remove commented-out code in `primitives/consensus/beefy/src/commitment.rs`

Co-authored-by: Davide Galassi <[email protected]>

* Fix inconsistency in panic message in  `primitives/io/src/lib.rs`

Co-authored-by: Davide Galassi <[email protected]>

* Remove redundant feature activation in `primitives/io/Cargo.toml`

Co-authored-by: Davide Galassi <[email protected]>

* - make `w3f-bls` a dev-dependancy only for beefy primitives.

- clean up comments.

Co-authored-by: Davide Galassi <[email protected]>

* export BEEFY KEY_TYPE from primitives/consensus/beefy
make `frame/consensus/beefy` in dependent of sp_crypto_app
use consistent naming in the beefy primitive tests.

* - implement `BeefyAuthorityId` for `bls_crypto::AuthorityId`.
- implement `bls_verify_works` test for BEEFY `bls_crypto`.

* Remove BEEFY `ecdsa_n_bls_crypto` for now for later re-introduction

* Make commitment and witness BEEFY tests not use Keystore.

* put `bls_beefy_verify_works` test under `bls-experimental` flag.

* bump up Runtime `BeefyAPI` to version 3 due to introducing generic AuthorityId.

* reuse code and encapsulate w3f-bls backend in sp-core as most as possible

Co-authored-by: Davide Galassi <[email protected]>

* Make comments in primities BEEFY `commitment.rs` and `witness.rs``tests convention conforming

* Use master dep versions

* Trivial change. Mostly to trigger CI

* Apply suggestions from code review

Co-authored-by: André Silva <[email protected]>

* Fix Cargo.toml

* Trigger CI with cumulus companion

* Trigger CI after polkadot companion change

---------

Co-authored-by: Davide Galassi <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Fix std, runtime-benchmarks and try-runtime features

zepter lint propagate-feature --feature try-runtime --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="try-runtime:frame-try-runtime"
zepter lint propagate-feature --feature runtime-benchmarks --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="runtime-benchmarks:frame-benchmarking"
zepter lint propagate-feature --feature std --left-side-feature-missing=ignore --workspace --fix

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add propagate feature CI check

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Test CI by adding an error

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use --locked

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add help msg

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Test CI by adding an error"

This reverts commit cf4ff6cc0632269b0a109e547686e5e3314b02de.

* Test CI by adding an error

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* No newline in help msg

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Test CI by adding an error"

This reverts commit 5daa06ada8e01f5bebafb9d1c76804dd79bc1006.

* Test CI by adding an error

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Test CI by adding an error"

This reverts commit ca15de5729507a564f140a10ec2e87b19516ec4c.

* Fix msg

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert back to master

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Re-do with Zepter v0.7.4

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update Zepter to 0.7.4

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Disable rococo try-runtime check

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* More review fixes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* add doc-only substrate entry point crate

* document a few more things

* add more

* fix width

* Update primitives/io/src/lib.rs

Co-authored-by: Gonçalo Pestana <[email protected]>

* add link

* update cargo toml file

* fix sp-io docs

* improve

* small update

* add license

* satisfy license job

* add a line about FRAME

* CI happy now

* make CI more happy

* Let the check run for the whole workspace

* Forward the substrate node again as default run

* update binary names

* upate verison test

* Fix fix fix

* Fix

* rename to substrate-node in more places

* Revert "rename to substrate-node in more places"

This reverts commit 66960f84a1b6f1f7c638b4040e28e9fbabb8adf5.

* fix

* Fix build pipeline

* Fix properly plus add some docs

---------

Co-authored-by: Gonçalo Pestana <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Added ECIES encryption

* tweaks

* fmt

* Make clippy happy

* Use local keystore

* qed

Signed-off-by: Alexandru Vasile <[email protected]>

Signed-off-by: Alexandru Vasile <[email protected]>

This removes the deprecated batch verification. This was actually never really activated.
Nevertheless, we need to keep the host functions around to support old runtimes which may import
these host functions. However, we do not give access to these functions anymore. This means that any new
runtime can not call them anymore. The host function implementations we keep will not do batch verification and will
instead fall back to the always existing option of directly verifying the passed signature.
`finish_batch_verification` will return the combined result of all the batch verify calls.

This removes the `TaskExecutorExt` which only existed to support the batch verification. So, any
code that used this extension can just remove the registration of them. It also removes
`SignatureBatching` that was used by `frame-executive` to control the batch verification.
However, there wasn't any `Verify` implementation that called the batch verification functions.

* Support stable rust for compiling the runtime

This pull request brings support for compiling the runtime with stable Rust. This requires at least
rust 1.68.0 to work on stable. The code is written in a way that it is backwards compatible and
should automatically work when someone compiles with 1.68.0+ stable.

* We always support nightlies!

* 🤦



* Sort by version

* Review feedback

* Review feedback

* Fix version parsing

* Apply suggestions from code review

Co-authored-by: Koute <[email protected]>

---------

Co-authored-by: Koute <[email protected]>

* Introduce keystore specialized sign methods

* Get rid of 'AppKey::UntypedGeneric' associated type.

Untyped generics are accessible using associated types 'Generic' associated type.
I.e. <T as AppKey>::Public::Generic

* Get rid of 'CryptoTypePublicPair'

* Trivial fix

* Small refactory of local keystore implementations

* Remove 'crypto_id' method from 'Public'

* Trivial rename of 'AppKey' to 'AppCrypto'

* Remove unused import

* Improve docs

* Better signature related errors for authority-discovery

* Apply review suggestion

* Apply review suggestions

Co-authored-by: Koute <[email protected]>

* Authority discoverty signing error revisited

* Signing error revisited for babe and aura as well

* Further cleanup

---------

Co-authored-by: Koute <[email protected]>

* Remove bloat about remote keystore

* Update docs and remove unused 'KeystoreRef' trait

* Use wherever possible, MemoryKeystore for testing

* Remove unrequired fully qualified method syntax for Keystore

* Remove 'supported_keys' 'sign_with_any' and 'sign_with_all' from keystore trait

* Remove the aync keystore

* Renaming:
- SyncCryptoStore -> Keystore
- SyncCryptoStorePtr -> KeystorePtr
- KeyStore -> MemoryKeystore

* Fix authority discovery worker and tests

* Rename 'insert_unknown' to 'insert'

* Remove leftover

* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh

* Make DispatchError impl MEL

* Upgrade SCALE codec to support `codec(skip)` for MEL

Co-authored-by: Bastian Köcher <[email protected]>

* ed25519: Don't panic for invalid signature

We should not panic for an invalid signature when the `UseDalekExt` is given.

* Update Cargo.toml

* Update primitives/io/Cargo.toml

* *: update rand to v0.8.5

* *: remove useless deps from Cargo.toml

* fix pallet-session-benchmarking

* fix pallet-election-provider-support test

* remove useless rand from dev-dependencies

* Remove sandboxing interface

* Remove unused struct

* ed25519_verify: Support using dalek for historical blocks

The switch from `ed25519-dalek` to `ed25519-zebra` was actually a breaking change. `ed25519-zebra`
is more permissive. To support historical blocks when syncing a chain this pull request introduces
an externalities extension `UseDalekExt`. This extension is just used as a signaling mechanism to
`ed25519_verify` to use `ed25519-dalek` when it is present. Together with `ExtensionBeforeBlock` it
can be used to setup a node in way to sync historical blocks that require `ed25519-dalek`, because
they included a transaction that verified differently as when using `ed25519-zebra`.

This feature can be enabled in the following way. In the chain service file, directly after the
client is created, the following code should be added:

```
use sc_client_api::ExecutorProvider;
client.execution_extensions().set_extensions_factory(
	sc_client_api::execution_extensions::ExtensionBeforeBlock::<Block, sp_io::UseDalekExt>::new(BLOCK_NUMBER_UNTIL_DALEK_SHOULD_BE_USED)
);
```

* Fix doc

* More fixes

* Update client/api/src/execution_extensions.rs

Co-authored-by: André Silva <[email protected]>

* Fix merge and warning

* Fix docs

Co-authored-by: André Silva <[email protected]>

* chore(release): sp-core v7.0.0

* chore(release): sp-runtime v7.0.0

* fix bad merge

* Remove `sp_tasks::spawn` API and related code

* Remove `RuntimeTasks::{spawn, join}` host functions

* remove unused

* Remove a few more tests that I forgot to remove

Co-authored-by: Shawn Tabrizi <[email protected]>

* Add std feature

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* WIP

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Impl function also in tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix sp-trie feature

Something makes the bench regression guard fail, maybe this?

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add runtime-benchmarks feature to sc-service

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Fix sp-trie feature"

This reverts commit f2cddfe41bc72e6f2f8133795ec9408ba0c3ec63.
Was already fixed, only needed a CI retry.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>