It doesn't make sense to only reorder the features array.

For example:

This makes it hard for me to compare the dependencies and features,
especially some crates have a really really long dependencies list.
```toml​
[dependencies]
c = "*"
a = "*"
b = "*"

[features]
std = [
  "a",
  "b",
  "c",
]
```

This makes my life easier.
```toml​
[dependencies]
a = "*"
b = "*"
c = "*"

[features]
std = [
  "a",
  "b",
  "c",
]
```

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>

Close: #5858

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

This PR adapts the `archive_storage` implementation from a method to a
subscription.

This keeps the archive APIs uniform and consistent.

Builds on: https://github.com/paritytech/polkadot-sdk/pull/5997

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>

# Description

At moonbeam we have worked on a `lazy-loading` feature which is a client
mode that forks a live parachain and fetches its state on-demand, we
have been able to do this by duplicating some code from
`sc_service::client`. The objective of this PR is to simplify the
implementation by making public some types in polkadot-sdk.

- Modules:
- `sc_service::client` **I do not see a point to only expose this type
when `test-helpers` feature is enabled**

## Integration

Not applicable, the PR just makes some types public.

## Review Notes

The changes included in this PR give more flexibility for client
developers by exposing important types.

This PR implements the `archive_unstable_storageDiff`.

The implementation follows the rpc-v2 spec from:
-  https://github.com/paritytech/json-rpc-interface-spec/pull/159.
- builds on top of
https://github.com/paritytech/json-rpc-interface-spec/pull/161

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>

Before this was done for every imported transaction. When a lot of
transactions got imported, the import notification channel was filled.
The underlying problem was that the `status` call is read locking the
`validated_pool` which will be write locked by the internal submitting
logic. Thus, the submitting and status reading was interferring which
each other.

---------

Co-authored-by: GitHub Action <action@github.com>

This PR refactors `TransactionPool` API to use `async_trait`, replacing
the` Pin<Box<...>>` pattern. This should improve readability and
maintainability.

The change is not altering any functionality.

---------

Co-authored-by: GitHub Action <action@github.com>

# Description

This seems to be an old artifact of the long closed
https://github.com/paritytech/substrate/issues/6827 that I noticed when
working on related code earlier.

## Integration

`NetworkStarter` was removed, simply remove its usage:
```diff
-let (network, system_rpc_tx, tx_handler_controller, start_network, sync_service) =
+let (network, system_rpc_tx, tx_handler_controller, sync_service) =
    build_network(BuildNetworkParams {
...
-start_network.start_network();
```

## Review Notes

Changes are trivial, the only reason for this to not be accepted is if
it is desired to not start network automatically for whatever reason, in
which case the description of network starter needs to change.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

# Description

This is a continuation of
https://github.com/paritytech/polkadot-sdk/pull/5666 that finally fixes
https://github.com/paritytech/polkadot-sdk/issues/5333.

This should allow developers to create custom syncing strategies or even
the whole syncing engine if they so desire. It also moved syncing engine
creation and addition of corresponding protocol outside
`build_network_advanced` method, which is something Bastian expressed as
desired in
https://github.com/paritytech/polkadot-sdk/issues/5#issuecomment-1700816458

Here I replaced strategy-specific types and methods in `SyncingStrategy`
trait with generic ones. Specifically `SyncingAction` is now used by all
strategies instead of strategy-specific types with conversions.
`StrategyKey` was an enum with a fixed set of options and now replaced
with an opaque type that strategies create privately and send to upper
layers as an opaque type. Requests and responses are now handled in a
generic way regardl...

Various fixes for the release of eth-rpc & ah-westend-runtime

- Bump asset-hub westend spec version
- Fix the status of the Receipt to properly report failed transactions
- Fix value conversion between native and eth decimal representation

---------

Co-authored-by: GitHub Action <action@github.com>

### Fork-Aware Transaction Pool Implementation

This PR introduces a fork-aware transaction pool (fatxpool) enhancing
transaction management by maintaining the valid state of txpool for
different forks.

### High-level overview
The high level overview was added to
[`sc_transaction_pool::fork_aware_txpool`](https://github.com/paritytech/polkadot-sdk/blob/3ad0a1b7/substrate/client/transaction-pool/src/fork_aware_txpool/mod.rs#L21)
module. Use:
```
cargo  doc --document-private-items -p sc-transaction-pool --open
```
to build the doc. It should give a good overview and nice entry point
into the new pool's mechanics.

<details>
  <summary>Quick overview (documentation excerpt)</summary>

#### View
For every fork, a view is created. The view is a persisted state of the
transaction pool computed and updated at the tip of the fork. The view
is built around the existing `ValidatedPool` structure.

A view is created on every new best block notification. To create a
view, one of the existing views is chosen and cloned.

When the chain progresses, the view is kept in the cache
(`retracted_views`) to allow building blocks upon intermediary blocks in
the fork.

The views are deleted on finalization: views lower than the finalized
block are removed.

The views are updated with the transactions from the mempool—all
transactions are sent to the newly created views.
A maintain process is also executed for the newly created
views—basically resubmitting and pruning transactions from the
appropriate tree route.

##### View store
View store is the helper structure that acts as a container for all the
views. It provides some convenient methods.

##### Submitting transactions
Every transaction is submitted to every view at the tips of the forks.
Retracted views are not updated.
Every transaction also goes into the mempool.

##### Internal mempool
Shortly, the main purpose of an internal mempool is to prevent a
transaction from being lost. That could happen when a transaction is
invalid on one fork and could be valid on another. It also allows the
txpool to accept transactions when no blocks have been reported yet.

The mempool removes its transactions when they get finalized.
Transactions are also periodically verified on every finalized event and
removed from the mempool if no longer valid.

#### Events
Transaction events from multiple views are merged and filtered to avoid
duplicated events.
`Ready` / `Future` / `Inblock` events are originated in the Views and
are de-duplicated and forwarded to external listeners.
`Finalized` events are originated in fork-aware-txpool logic.
`Invalid` events requires special care and can be originated in both
view and fork-aware-txpool logic.

#### Light maintain
Sometime transaction pool does not have enough time to prepare fully
maintained view with all retracted transactions being revalidated. To
avoid providing empty ready transaction set to block builder (what would
result in empty block) the light maintain was implemented. It simply
removes the imported transactions from ready iterator.

#### Revalidation
Revalidation is performed for every view. The revalidation process is
started after a trigger is executed. The revalidation work is terminated
just after a new best block / finalized event is notified to the
transaction pool.
The revalidation result is applied to the newly created view which is
built upon the revalidated view.

Additionally, parts of the mempool are also revalidated to make sure
that no transactions are stuck in the mempool.


#### Logs
The most important log allowing to understand the state of the txpool
is:
```
              maintain: txs:(0, 92) views:[2;[(327, 76, 0), (326, 68, 0)]] event:Finalized { hash: 0x8...f, tree_route: [] }  took:3.463522ms
                             ^   ^         ^     ^   ^  ^      ^   ^  ^        ^                                                   ^
unwatched txs in mempool ────┘   │         │     │   │  │      │   │  │        │                                                   │
   watched txs in mempool ───────┘         │     │   │  │      │   │  │        │                                                   │
                     views  ───────────────┘     │   │  │      │   │  │        │                                                   │
                      1st view block # ──────────┘   │  │      │   │  │        │                                                   │
                           number of ready tx ───────┘  │      │   │  │        │                                                   │
                                numer of future tx ─────┘      │   │  │        │                                                   │
                                        2nd view block # ──────┘   │  │        │                                                   │
                                      number of ready tx ──────────┘  │        │                                                   │
                                           number of future tx ───────┘        │                                                   │
                                                                 event ────────┘                                                   │
                                                                       duration  ──────────────────────────────────────────────────┘
```
It is logged after the maintenance is done.

The `debug` level enables per-transaction logging, allowing to keep
track of all transaction-related actions that happened in txpool.
</details>


### Integration notes

For teams having a custom node, the new txpool needs to be instantiated,
typically in `service.rs` file, here is an example:

https://github.com/paritytech/polkadot-sdk/blob/9c547ff3

/cumulus/polkadot-omni-node/lib/src/common/spec.rs#L152-L161

To enable new transaction pool the following cli arg shall be specified:
`--pool-type=fork-aware`. If it works, there shall be information
printed in the log:
```
2024-09-20 21:28:17.528  INFO main txpool: [Parachain]  creating ForkAware txpool.
````

For debugging the following debugs shall be enabled:
```
      "-lbasic-authorship=debug",
      "-ltxpool=debug",
```
*note:* trace for txpool enables per-transaction logging.

### Future work
The current implementation seems to be stable, however further
improvements are required.
Here is the umbrella issue for future work:
- https://github.com/paritytech/polkadot-sdk/issues/5472


Partially fixes: #1202

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Iulian Barbu <14218860+iulianbarbu@users.noreply.github.com>

cc https://github.com/paritytech/ci_cd/issues/1035

cc https://github.com/paritytech/ci_cd/issues/1023

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>
Co-authored-by: gui <gui.thiolliere@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

# Description

Follow-up to https://github.com/paritytech/polkadot-sdk/pull/5469 and
mostly covering https://github.com/paritytech/polkadot-sdk/issues/5333.

The primary change here is that syncing strategy is no longer created
inside of syncing engine, instead syncing strategy is an argument of
syncing engine, more specifically it is an argument to `build_network`
that most downstream users will use. This also extracts addition of
request-response protocols outside of network construction, making sure
they are physically not present when they don't need to be (imagine
syncing strategy that uses none of Substrate's protocols in its
implementation for example).

This technically allows to completely replace syncing strategy with
whatever strategy chain might need.

There will be at least one follow-up PR that will simplify
`SyncingStrategy` trait and other public interfaces to remove mentions
of block/state/warp sync requests, replacing them with generic APIs,
such that strategies where warp sync is not applicable don't have to
provide dummy method implementations, etc.

## Integration

Downstream projects will have to write a bit of boilerplate calling
`build_polkadot_syncing_strategy` function to create previously default
syncing strategy.

## Review Notes

Please review PR through individual commits rather than the final diff,
it will be easier that way. The changes are mostly just moving code
around one step at a time.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)

This feature is helpful for us with custom sync protocol that is similar
to Warp sync except we do not ever sync the gap and don't want it to
exist in the first place (see
https://github.com/paritytech/polkadot-sdk/issues/5333 and its
references for motivation).

Otherwise we had to resort to this:
https://github.com/autonomys/polkadot-sdk/commit/d5375125ca7c59fcc1ac72dc1b2ac251cbc80323

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>

# Description

This PR allows the RPC server's socket address to be returned when
initializing the server. This allows the library consumer to easily
programmatically determine which port the RPC server is listening on. My
use case for this is automated testing. I'd like to be able to simply
specify that the server bind to port '0' and then test against whatever
port the OS assigns dynamically. I will have many RPC servers running in
parallel across many tests within a single process, and I don't want to
have to deal with port conflicts.

## Integration

Integration is straightforward. My main concern is that I am making
non-backwards-compatible changes to public library functions. Let me
know if I should leave backwards-compatible wrappers in place for
any/all of the public functions that were modified.

## Review Notes
The rationale for making the new listen_addresses field on the
RpcHandlers struct a ```[MultiAddr]``` rather than ```SocketAddr``` is
because I wanted it to be transport-agnostic as well as capable of
supporting multiple listening addresses in case that is ever required by
the RPC server in the future.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

1. I didn't understand what the 'T' label meant. Am I supposed to open a
github Issue for my PR?
2. I didn't see an easy way to add tests since the functions I am
modifying are not directly called by any tests.

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

Previously, block gaps could only be created by warp sync, but block
gaps will also be generated by fast sync once #5406 is fixed. This PR is
part 1 of the detailed implementation plan in
https://github.com/paritytech/polkadot-sdk/issues/5406#issuecomment-2325064863:
refactor `BlockGap`.

This refactor converts the existing `(NumberFor<Block>,
NumberFor<Block>)` into a dedicated `BlockGap<NumberFor<Block>>` struct.
This change is purely structural and does not alter existing logic, but
lays the groundwork for the follow-up PR.

The compatibility concern caused by the new structure is addressed in
the second commit.

cc @dmitry-markin



---------

Co-authored-by: Bastian Köcher <git@kchr.de>

This improves `sc-service` API by not requiring the whole
`&Configuration`, using specific configuration options instead.
`RpcConfiguration` was also extracted from `Configuration` to group all
RPC options together.

We don't use Substrate's CLI and would rather not use `Configuration`
either, but some key public functions require it even though they
ignored most of the fields anyway.

`RpcConfiguration` is very helpful not just for consolidation of the
fields, but also to finally make RPC optional for our use case, while
Substrate still runs RPC server on localhost even if listening address
is explicitly set to `None`, which is annoying (and I suspect there is a
reason for it, so didn't want to change the default just yet).

While this is a breaking change, most developers will not notice it if
they use higher-level APIs.

Fixes https://github.com/paritytech/polkadot-sdk/issues/2897

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

rpc server: listen to `ipv6 socket` if available and `--experimental-rpc-endpoint` CLI option (#4792)

Close https://github.com/paritytech/polkadot-sdk/issues/3488,
https://github.com/paritytech/polkadot-sdk/issues/4331

This changes/adds the following:

1. The default setting is that substrate starts a rpc server that
listens to localhost both Ipv4 and Ipv6 on the same port. Ipv6 is
allowed to fail because some platforms may not support it
2. A new RPC CLI option `--experimental-rpc-endpoint` which allow to
configure arbitrary listen addresses including the port, if this is
enabled no other interfaces are enabled.
3. If the local addr is not found for any of the sockets the server is
not started throws an error.
4. Remove the deny_unsafe from the RPC implementations instead this is
an extension to allow different polices for different interfaces/sockets
such one may enable unsafe on local interface and safe on only the
external interface.

So for instance in this PR it's now possible to start up three RPC...

As I was looking at the coupling between `SyncingEngine`,
`SyncingStrategy` and individual strategies I noticed a few things that
were unused, redundant or awkward.

The awkward change comes from
https://github.com/paritytech/substrate/pull/13700 where
`num_connected_peers` property was added to `SyncStatus` struct just so
it can be rendered in the informer. While convenient, the property
didn't really belong there and was annoyingly set to `0` in some
strategies and to `num_peers` in others. I have replaced that with a
property on `SyncingService` that already stored necessary information
internally.

Also `ExtendedPeerInfo` didn't have a working `Clone` implementation due
to lack of perfect derive in Rust and while I ended up not using it in
the refactoring, I included fixed implementation for it in this PR
anyway.

While these changes are not strictly necessary for
https://github.com/paritytech/polkadot-sdk/issues/5333, they do reduce
coupling of syncing engine with syncing strategy, which I thought is a
good thing.

Reviewing individual commits will be the easiest as usual.

---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

I'm not sure if this is exactly what
https://github.com/paritytech/polkadot-sdk/issues/3537 meant, but I
think it should be fine to wait for relay chain before initializing
parachain node fully, which removed the need for background task and
extra hacks throughout the stack just to know where warp sync should
start.

Previously there were both `WarpSyncParams` and `WarpSyncConfig`, but
there was no longer any point in having two data structures, so I
simplified it to just `WarpSyncConfig`.

Fixes https://github.com/paritytech/polkadot-sdk/issues/3537

There was no need for it to be `&mut self` since block import can happen
concurrently for different blocks and in many cases it was `&mut Arc<dyn
BlockImport>` anyway :man_shrugging:

Similar in nature to
https://github.com/paritytech/polkadot-sdk/pull/4844

The same exact `matches!()` was duplicated in
`Configuration::no_genesis()` method and inline in full node parts
creation. Since this is the same exact logic and reason, it makes sense
to de-duplicate them.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Follow-up to https://github.com/paritytech/polkadot-sdk/pull/4457, looks
like more things were missing

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

This PR enables the `chainSpec_v1` class for the polkadot-parachian. 
The chainSpec is part of the rpc-v2 which is spec-ed at:
https://github.com/paritytech/json-rpc-interface-spec/blob/main/src/api/chainSpec.md.

This also paves the way for enabling a future `chainSpec_unstable_spec`
on all nodes.

Closes: https://github.com/paritytech/polkadot-sdk/issues/5191

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

A bit of a controversial move, but a good preparation for even further
reducing the traffic on outdated content of `substrate.io`. Current
status:

<img width="728" alt="Screenshot 2024-07-15 at 11 32 48"
src="https://github.com/user-attachments/assets/df33b164-0ce7-4ac4-bc97-a64485f12571">

Previously, I was in favor of changing the domain of the rust-docs to
something like `polkadot-sdk.parity.io` or similar, but I think the
current format is pretty standard and has a higher chance of staying put
over the course of time:

`<org-name>.github.io/<repo-name>` ->
`https://paritytech.github.io/polkadot-sdk/`

part of https://github.com/paritytech/eng-automation/issues/10

Now Polkadot-SDK is ansi_term free

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

`polkadot-parachain` simplifications and deduplications

Details in the commit messages. Just copy-pasting the last commit
description since it introduces the biggest changes:

```
    Implement a more structured way to define a node spec
    
    - use traits instead of bounds for `rpc_ext_builder()`,
      `build_import_queue()`, `start_consensus()`
    - add a `NodeSpec` trait for defining the specifications of a node
    - deduplicate the code related to building a node's components /
      starting a node
```

The other changes are much smaller, most of them trivial and are
isolated in separate commits.

A few refactorings to block import and block verification that should
not be controversial.

Block verification before block import is stateless by design as
described in https://substrate.stackexchange.com/a/1322/25 and the fact
that it wasn't yet I consider to be a bug. Some code that requires it
had to use `Mutex`, but I do not expect it to have a measurable
performance impact.

Similarly with block import checking whether block preconditions should
not be an exclusive operation, there is nothing fundamentally wrong with
checking a few competing blocks whose parent blocks exist at the same
time (and even import them concurrently later, though IIRC this is not
yet implemented either).

They were originally a part of
https://github.com/paritytech/polkadot-sdk/pull/4842 and upstreaming
will help us to reduce the size of the patch we need to apply on top of
upstream code at Subspace every time we upgrade. There are no new
features introduced here, just refactoring to get rid of unnecessary
requirements.

After preparing in https://github.com/paritytech/polkadot-sdk/pull/4633,
we can lift also all internal dependencies up to the workspace.

This does not actually change anything, but uses `workspace = true` for
all dependencies. You can check it with:
```bash
git checkout -q $(git merge-base oty-lift-all-deps origin/master)
cargo tree -e features > master.out

git checkout -q oty-lift-all-deps
cargo tree -e features > new.out
diff master.out new.out
```

It did not yet lift 100% of dependencies, some inside of `target.*` or
some that had conflicting aliases introduced recently. But i will do
these together in a follow-up with CI checks.

Can be reproduced with [zepter](https://github.com/ggwpez/zepter/):
`zepter transpose d lift-to-workspace "regex:.*" --version-resolver
highest --skip-package "polkadot-sdk" --ignore-errors --fix`.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

This unifies the logic between `CallExecutor` and `Client` when it comes
to fetching the `code` for a given block. The actual `code` depends on
potential overrides/substitutes.

Besides that it changes the logic in the lookahead collator on which
`ValidationCodeHash` it sends to the validator alongside the `POV`. We
are now sending the code hash as found on the relay chain. This is done
as the local node could run with an override which is compatible to the
validation code on the relay chain, but has a different hash.

Related to: https://github.com/paritytech/polkadot-sdk/issues/5

A couple of cosmetics and improvements related to
`polkadot-parachain-bin`:

- Adding some convenience traits in order to avoid declaring long
duplicate bounds
- Specifically check if the runtime exposes `AuraApi` when executing
`start_lookahead_aura_consensus()`
- Some fixes for the `RelayChainCli`. Details in the commits description

Inherited workspace dependencies cannot be renamed by the crate using
them (see [1](https://github.com/rust-lang/cargo/issues/12546),
[2](https://stackoverflow.com/questions/76792343/can-inherited-dependencies-in-rust-be-aliased-in-the-cargo-toml-file)).
Since we want to use inherited workspace dependencies everywhere, we
first need to unify all aliases that we use for a dependency throughout
the workspace.
The umbrella crate is currently excluded from this procedure, since it
should be able to export the crates by their original name without much
hassle.

For example: one crate may alias `parity-scale-codec` to `codec`, while
another crate does not alias it at all. After this change, all crates
have to use `codec` as name. The problematic combinations were:
- conflicting aliases: most crates aliases as `A` but some use `B`.
- missing alias: most of the crates alias a dep but some dont.
- superfluous alias: most crates dont alias a dep but some do.

The script that i used first determines whether most crates opted to
alias a dependency or not. From that info it decides whether to use an
alias or not. If it decided to use an alias, the most common one is used
everywhere.

To reproduce, i used
[this](https://github.com/ggwpez/substrate-scripts/blob/master/uniform-crate-alias.py)
python script in combination with
[this](https://github.com/ggwpez/zepter/blob/38ad10585fe98a5a86c1d2369738bc763a77057b/renames.json)
error output from Zepter.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

This PR removes deprecated code:
- The `RuntimeGenesisConfig` generic type parameter in
`GenericChainSpec` struct.
- `ChainSpec::from_genesis` method allowing to create chain-spec using
closure providing runtime genesis struct
- `GenesisSource::Factory` variant together with no longer needed
`GenesisSource`'s generic parameter `G` (which was intended to be a
runtime genesis struct).


https://github.com/paritytech/polkadot-sdk/blob/17b56fae/substrate/client/chain-spec/src/chain_spec.rs#L559-L563

This bump of versions:

https://github.com/paritytech/polkadot-sdk/pull/4409/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87eR11936

reintroduced a dependency to proc-macro-crate 2.0.0 which is suffering
from: https://github.com/bkchr/proc-macro-crate/pull/42 this, so bump
parity-scale-codec to a newer version to eliminate the bad
proc-macro-crate 2.0.0 dependency.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: command-bot <>

https://github.com/paritytech/polkadot-sdk/pull/3166 made private
functions used in `spawn_tasks()` public but forgot to add them in
exported functions of the crate.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

This PR changes the fork calculation and pruning algorithm to enable
future block header pruning. It's required because the previous
algorithm relied on the block header persistence. It follows the
[related
discussion](https://github.com/paritytech/polkadot-sdk/issues/1570)

The previous code contained this comment describing the situation:
```
	/// Note a block height finalized, displacing all leaves with number less than the finalized
	/// block's.
	///
	/// Although it would be more technically correct to also prune out leaves at the
	/// same number as the finalized block, but with different hashes, the current behavior
	/// is simpler and our assumptions about how finalization works means that those leaves
	/// will be pruned soon afterwards anyway.
	pub fn finalize_height(&mut self, number: N) -> FinalizationOutcome<H, N> {
```

The previous algorithm relied on the existing block headers to prune
forks later and to enable block header pruning we need to clear all
obsolete forks right after the block finalization to not depend on the
related block headers in the future.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

This PR adds two new CLI options to disable rate limiting for certain ip
addresses and whether to trust "proxy header".
After going back in forth I decided to use ip addr instead host because
we don't want rely on the host header which can be spoofed but another
solution is to resolve the ip addr from the socket to host name.

Example:

```bash
$ polkadot --rpc-rate-limit 10 --rpc-rate-limit-whitelisted-ips 127.0.0.1/8 --rpc-rate-limit-trust-proxy-headers
```

The ip addr is read from the HTTP proxy headers `Forwarded`,
`X-Forwarded-For` `X-Real-IP` if `--rpc-rate-limit-trust-proxy-headers`
is enabled if that is not enabled or the headers are not found then the
ip address is read from the socket.

//cc @BulatSaif can you test this and give some feedback on it?