Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: command-bot <>

This improves `sc-service` API by not requiring the whole
`&Configuration`, using specific configuration options instead.
`RpcConfiguration` was also extracted from `Configuration` to group all
RPC options together.

We don't use Substrate's CLI and would rather not use `Configuration`
either, but some key public functions require it even though they
ignored most of the fields anyway.

`RpcConfiguration` is very helpful not just for consolidation of the
fields, but also to finally make RPC optional for our use case, while
Substrate still runs RPC server on localhost even if listening address
is explicitly set to `None`, which is annoying (and I suspect there is a
reason for it, so didn't want to change the default just yet).

While this is a breaking change, most developers will not notice it if
they use higher-level APIs.

Fixes https://github.com/paritytech/polkadot-sdk/issues/2897

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

Gensis config presets moved from `parachain-template-node` binary into
`parachain-template-runtime` runtime.

cc: @PierreBesson

Tiny changes to simplify the code:

- Remove an unnecessary `collect`.
- Reduce the code duplication a little bit.

---------

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

closes https://github.com/paritytech/polkadot-sdk/issues/5448

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

# Description

Add `starting_timestamp` function for `Slot` type.

## Integration

This is an addition of public function to a type, so integration should
be seamless for idiomatic use of Rust.

## Review Notes

Since `Slot` is just a slot number, the it's starting timestamp depends
on `SlotDuration` which is a parameter to the added function. This
function can be seen as dual to existing `fn from_timestamp`.
Because there is a potential for overflow, the return type is `Option`.

Q1: should I introduce tests for in this crate and add cases for both
case: overflow (`None`) and no overflow (`Some`)?

Q2: How can I add labels? IMO they should be `T0-node` and `D0-easy` but
I cannot add them using GH interface.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

---------

Co-authored-by: Squirrel <giles.cope@iohk.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>

Currently, when the pallet is compiled with the `insecure_zero_ed flag`,
benchmarks fail because the minimum balance is set to zero.

The PR aims to resolve this issue by implementing a placeholder value
for the minimum balance when the `insecure_zero_ed` flag is active. it
ensures that benchmarks run successfully regardless of whether this flag
is used or not

ISSUE
Link to the issue:
https://github.com/paritytech/polkadot-sdk/issues/3326

Deliverables

[Deprecation] remove pallet::getter usage from pallet-treasury

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Dónal Murray <donalm@seadanda.dev>
Co-authored-by: Bastian Köcher <git@kchr.de>

A test is triggering a log error. But is correct and successful. This is
a refactor without triggering the log error.

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

rpc server: listen to `ipv6 socket` if available and `--experimental-rpc-endpoint` CLI option (#4792)

Close https://github.com/paritytech/polkadot-sdk/issues/3488,
https://github.com/paritytech/polkadot-sdk/issues/4331

This changes/adds the following:

1. The default setting is that substrate starts a rpc server that
listens to localhost both Ipv4 and Ipv6 on the same port. Ipv6 is
allowed to fail because some platforms may not support it
2. A new RPC CLI option `--experimental-rpc-endpoint` which allow to
configure arbitrary listen addresses including the port, if this is
enabled no other interfaces are enabled.
3. If the local addr is not found for any of the sockets the server is
not started throws an error.
4. Remove the deny_unsafe from the RPC implementations instead this is
an extension to allow different polices for different interfaces/sockets
such one may enable unsafe on local interface and safe on only the
external interface.

So for instance in this PR it's now possible to start up three RPC
endpoints as follows:
```
$ polkadot --experimental-rpc-endpoint "listen-addr=127.0.0.1:9944,rpc-methods=unsafe" --experimental-rpc-endpoint "listen-addr=0.0.0.0:9945,rpc-methods=safe,rate-limit=100" --experimental-rpc-endpoint "listen-addr=[::1]:9944,optional=true"
```

#### Needs to be addressed

~1. Support binding to a random port if it's fails with the default
stuff for backward compatible reasons~
~2. How to sync that the rpc CLI params and that the rpc-listen-addr
align, hard to maintain...~
~3. Add similar warning prints for exposing unsafe methods on external
interfaces..~
~4. Inline todos + the hacky String conversion from rpc params.~

#### Cons with this PR

Manual strings parsing impl more error-prone than relying on clap....

//cc @jsdw @BulatSaif @PierreBesson @bkchr



---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

as part of paritytech/devops/3502, try-runtime nodes were migrated to a
new provider with a new domain address. The PR fixes all the references
to try-runtime nodes on rococo, westend, kusama and polkadot networks.

---------

Co-authored-by: ArshamTeymouri <ArshamTeymouri@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

Changes:
- Set default level to `Info` again. Seems like a dependency update set
it to something higher.
- Fix docs to not use `--locked` since we rely on dependency bumps via
cargo.
- Add README with rust docs.
- Fix bug where the node ignored `--heap-pages` argument.

You can test the `--heap-pages` bug by running this command on master
and then on this branch. Note that it should fail because of the very
low heap pages arg:
`cargo run --release --bin polkadot --features=runtime-benchmarks --
benchmark pallet --chain=dev --steps=10 --repeat=30
--wasm-execution=compiled --heap-pages=8 --pallet=frame-system
--extrinsic="*"`

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

This can make the log cleaner, especially when you specify `--log
sync=debug`.

As I was looking at the coupling between `SyncingEngine`,
`SyncingStrategy` and individual strategies I noticed a few things that
were unused, redundant or awkward.

The awkward change comes from
https://github.com/paritytech/substrate/pull/13700 where
`num_connected_peers` property was added to `SyncStatus` struct just so
it can be rendered in the informer. While convenient, the property
didn't really belong there and was annoyingly set to `0` in some
strategies and to `num_peers` in others. I have replaced that with a
property on `SyncingService` that already stored necessary information
internally.

Also `ExtendedPeerInfo` didn't have a working `Clone` implementation due
to lack of perfect derive in Rust and while I ended up not using it in
the refactoring, I included fixed implementation for it in this PR
anyway.

While these changes are not strictly necessary for
https://github.com/paritytech/polkadot-sdk/issues/5333, they do reduce
coupling of syncing engine with syncing strategy, which I thought is a
good thing.

Reviewing individual commits will be the easiest as usual.

---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

Introduce `MaybeConsideration` extension trait for `Consideration`.

The trait allows for the management of tickets that may represent no
cost. While the `MaybeConsideration` still requires proper handling, it
introduces the ability to determine if a ticket represents no cost and
can be safely forgotten without any side effects.

The new trait is particularly useful when a consumer expects the cost to
be zero under certain conditions (e.g., when the proposal count is below
a threshold N) and does not want to store such consideration tickets in
storage. The extension approach allows us to avoid breaking changes to
the existing trait and to continue using it as a non-optional version
for migrating pallets that utilize the `Currency` and `fungible` traits
for `holds` and `freezes`, without requiring any storage migration.

# Description

Moves `create_inherent_data_provider` after checking if major sync is in
progress.

## Integration

Change is internal to sc-consensus-slots. It should be no-op unless
someone is using fork of this SDK.

## Review Notes

Motivation for this change is to avoid calling
`create_inherent_data_providers` if it's result is going to be discarded
anyway during major sync. This has potential to speed up node operations
during major sync by not calling possibly expensive
`create_inherent_data_provider`.

TODO: labels T0-node D0-simple
TODO: there is no tests for `Slots`, should I add one for this case?

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

Needed it downstream

This PR untangles syncing metrics and makes them reactive, the way
metrics are supposed to be in general.

Syncing metrics were bundled in a way that caused coupling across
multiple layers: justifications metrics were defined and managed by
`ChainSync`, but only updated periodically on tick in `SyncingEngine`,
while actual values were queried from `ExtraRequests`. This convoluted
architecture was hard to follow when I was looking into
https://github.com/paritytech/polkadot-sdk/issues/5333.

Now metrics that correspond to each component are owned by that
component and updated as changes are made instead of on tick every
1100ms.

This does add some annoying boilerplate that is a bit harder to
maintain, but it separates metrics more nicely and if someone queries
them more frequently will give arbitrary resolution. Since metrics
updates are just atomic operations I do not expect any performance
impact of these changes.

Will add prdoc if changes look good otherwise.

P.S. I noticed that importing requests (and corresponding metrics) were
not cleared ever since corresponding code was introduced in
https://github.com/paritytech/polkadot-sdk/commit/dc41558b#r145518721
and I left it as is to not change the behavior, but it might be
something worth fixing.

cc @dmitry-markin



---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

This is a heavily modified and stripped down version of
`pallet_contracts`. We decided to fork instead of extend the old pallet.
Reasons for that are:

- There is no benefit of supporting both on the same pallet as the
intended payload for the new pallet (recompiled YUL) will be using a
different ABI.
- It is much easier since it allows us to remove all the code that was
necessary to support Wasm and focus fully on running cross compiled YUL
contracts.

**The code is reviewable but can't be merged because it depends on an
unreleased version of PolkaVM via git.**

## Current state

All tests are passing and the code is not quick and dirty but written to
last. The work is not finished, though. It is included in the
`kitchensink-runtime` and a node can be built. However, we merge early
in order to be able to start testing other components as early as
possible.

Outstanding changes are tracked here and will be merged separately:
https://github.com/paritytech/polkadot-sdk/issues/5308

## Syscall Interface

The syscall interface is best explored by generating the docs of this
crate and looking at the `SyscallDoc` trait. Arguments are passed in
registers a0-a5 in the order they are listed. If there are more than 6
arguments (call, instantiate) a pointer to a packed struct of the
arguments is expected as the only argument. I plan to create variants of
those syscalls with less arguments specifically for YUL.

Functions are just referenced by their name as ASCII within the PolkaVM
container. Rather than by a syscall number as it was the case in the
last implementation.
 

## Changes vs. `pallet_contracts`

The changes are too numerous to list them all here. This is an
incomplete list:

- Use PolkaVM instead of wasmi to execute contracts
- Made Runtime generic over a new `Memory` trait as we can't map memory
directly on PolkaVM anymore
- No static verification on code upload. Everything is a determinstic
runtime failure
- Removed all migrations and reset the pallet version
- Removed the nonce storage item and instead use the deployers account
nonce to generate a unique trie
- We now bump the deployers account nonce on contract instantiation to
they are bumped even within a batch transaction
- Removed the instantiation nonce host function: We should add a new
`instantiate` variant as a replacement for thos
- ContractInfoOf of uses the indentity hasher now
- Remove the determinism feature: User of that feature should switch to
soft floats
- The `unstable` attribute has been replaced by a `api_version`
attribute to declare at which version an API became available
	- leaving out that attribute makes the API effectively unstable
- a new `api_version` field on the CodeInfo makes sure that old
contracts can't access new APIs (necessary due to lack of static
verification.
- Added a `behaviour_version` field to CodeInfo that can used if we need
to introduce breaking changes and keep the old behaviour for existing
contracts
- Unified storage vs. transient and fixed vs. variable sized keys all
into one set of multiplexing host functions
- Removed all contract observeable limits from the `Config` trait and
instead hardcode them
- Removed the Schedule
- Removed all deprecated host functions
- Simplify chain extension as preperation for making it a pre-compile

---------

Co-authored-by: command-bot <>

I'm not sure if this is exactly what
https://github.com/paritytech/polkadot-sdk/issues/3537 meant, but I
think it should be fine to wait for relay chain before initializing
parachain node fully, which removed the need for background task and
extra hacks throughout the stack just to know where warp sync should
start.

Previously there were both `WarpSyncParams` and `WarpSyncConfig`, but
there was no longer any point in having two data structures, so I
simplified it to just `WarpSyncConfig`.

Fixes https://github.com/paritytech/polkadot-sdk/issues/3537

This PR aims to make the logging from the peer store a bit more clear.

In the past, we aggressively produced warning logs from the peer store
component, even in cases where the reputation change was not malicious.
This has led to an extensive number of logs, as well to node operator
confusion.

In this PR, we produce a warning message if:
- The peer crosses the banned threshold for the first time. This is the
actual reason of a ban
- The peer misbehaves again while being banned. This may happen during a
batch peer report

cc @paritytech/networking 

Part of: https://github.com/paritytech/polkadot-sdk/issues/5379.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>

Just fixing the beefy primitives link to the right one after moving
these primitives to `consensus`

The CI isn't happy with the amount of output:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7035621/raw

---------

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

There was no need for it to be `&mut self` since block import can happen
concurrently for different blocks and in many cases it was `&mut Arc<dyn
BlockImport>` anyway :man_shrugging:

Similar in nature to
https://github.com/paritytech/polkadot-sdk/pull/4844

Fix https://github.com/paritytech/polkadot-sdk/issues/5184

`owner` can set himself as a `freezer` and freeze the asset so
requirement is not really needed. And requirement is not implemented.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

..without needing to provide your own `newtype` around it.  

This will allow `type Reason = u8` to be used as `FreezeReason` and
`HoldReason`, which I think is a nice simplification if one doens't want
to deal with the complications.

At the same time, it is a bit of an anti-pattern.

Putting it out there to check people's vibes.

fix #3631 

Types which are impacted and fixed here are `ItemTip`,
`PriceWithDirection`, `PreSignedMint`, `PreSignedAttributes`.

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

Update Identity pallet README.md according to the up-to-date docs,
particularly to explain the _username_ concept of the pallet.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

closes https://github.com/paritytech-secops/srlabs_findings/issues/408.
This fixes how ProxyDelegator accounts are derived but may cause issues
in Westend since it would use the old derivative accounts. Does not
affect Polkadot/Kusama as this pallet is not deployed to them yet.

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>

## Context
Pool members using the old `TransferStake` strategy were able to
transfer all their funds to the pool. With `DelegateStake` changes, we
want to ensure similar behaviour by allowing members to delegate all
their stake to the pool.

## Changes
- Ensure all the balance including ED of an account can be delegated
(and used in the pool) by adding a provider for delegators.
- Gates calls that mutates the pool or pool member if they are in
unmigrated state. Closes
https://github.com/paritytech-secops/srlabs_findings/issues/409.
- Adds remote test to migrate all pools and members to `DelegateStake`
which can be used with `Kusama` and `Polkadot` runtime state. closes
https://github.com/paritytech/polkadot-sdk/issues/4629.
- Add new runtime apis to read pool and member balance.

## Addressing possible migration errors 
Pool members migrating can run into two types of errors:
- Already Staking: If the pool member is already staking, we cannot
migrate them to `DelegateStake` since this may mean they are able to use
the same staked funds in the pool. Users would need to withdraw all
their funds from staking, in order to migrate their pool funds.
- Pool contribution below ED: For these cases transfer from pool account
to member account would fail. The affected users can top up their
accounts and redo migration.

Another error that was earlier possible was when member's free balance
is below ED. This PR adds a provider to delegator allowing all user
balance including ED can be contributed towards the pool. This helps
`1095` accounts in Polkadot and `41` accounts in Kusama to migrate now
which would have earlier failed.

## Results from RemoteExternalities Tests.

### Kusama
`Migration stats: success: 3017, direct_stakers: 361, unexpected_errors:
0`

### Polkadot
`Migration stats: success: 42859, direct_stakers: 643,
unexpected_errors: 0`

## TODO
- [x] Add runtime api for member total balance.
- [x] New
[issue](https://github.com/paritytech/polkadot-sdk/issues/5009) to reap
pool members with contribution below ED.
- [x] Add provider for delegators so whole balance including ED can be
held while contributing to pools.
- [x] Gate all pool extrinsics if pool/member is in non-migrated state.

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>

The same exact `matches!()` was duplicated in
`Configuration::no_genesis()` method and inline in full node parts
creation. Since this is the same exact logic and reason, it makes sense
to de-duplicate them.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Related to #4523 

This PR adds benchmarks for `report_fork_voting()`.

**Important: Even though the benchmarks are now available, we still use
`Weight::MAX`. That's because I realized while working on this PR that
there's still one missing piece. We should also check that the ancestry
proof is optimal. I plan to do this in a future PR, hopefully the last
one related to #4523.**

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: command-bot <>

Make ticket non-optional and add ensure_successful method to
Consideration trait.

Reverts the optional return ticket type for the new function introduced
in
[polkadot-sdk/4596](https://github.com/paritytech/polkadot-sdk/pull/4596)
and adds a helper `ensure_successful` function for the runtime
benchmarks.
Since the existing FRAME pallet represents zero cost with a zero balance
rather than `None` in an option, maintaining the ticket type as a
non-optional balance is beneficial for backward compatibility and helps
avoid unnecessary migrations.

In the move from XCMv3 to XCMv4, the `AssetId` for `ForeignAssets` in
`asset-hub-rococo` and `asset-hub-westend` was left as `v3::Location` to
be later migrated to `v4::Location`.

This is that migration PR.

Because the encoding of `v3::Location` and `v4::Location` is the same,
we don't need to do any data migration, the keys will still be
decodable.
The [original idea by
Jan](https://github.com/paritytech/polkadot/pull/7236) was to make the
v4 changes in v3 since the ABI (the encoding/decoding) didn't change.
Corroborated the ABI is the same iterating over all storage, the code is
on [another
branch](https://github.com/paritytech/polkadot-sdk/blob/cisco-assert-v3-v4-encodings-equal/cumulus/parachains/runtimes/assets/migrations/src/foreign_assets_to_v4/mod.rs).

We will need a data migration when we want to update from `v4::Location`
to `v5::Location` because of [the accepted RFC changing the NetworkId
enum](https://github.com/polkadot-fellows/RFCs/pull/108).
I'll configure MBMs (Multi-Block Migrations) then and make the actual
migration.

Fixes https://github.com/paritytech/polkadot-sdk/issues/4128

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

This PR performs minor code cleanup to reduce verbosity. Since the
compiler has already optimized out indirect calls in the existing code,
these changes improve readability but do not affect performance.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Resolves https://github.com/paritytech-secops/srlabs_findings/issues/412

## Changes
- Clear any dangling delegation when member is removed.
- Agents need to be killed explicitly when pools are destroyed.
- Member withdraw amount is max of their locked funds and the value of
their points.

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: command-bot <>

This PR adds an additional defensive check to the reclaim SE. 

Since it can happen that we miss some storage accesses on other SEs
pre-dispatch, we should double check
that the bookkeeping of the runtime stays ahead of the node-side
pov-size.

If we discover a mismatch and the node-side pov-size is indeed higher,
we should set the runtime bookkeeping to the node-side value. In cases
such as #5229, we would stop including extrinsics and not run `on_idle`
at least.

cc @gui1117

---------

Co-authored-by: command-bot <>

Hello
I found several spelling errors.
Br, Elias.