* [XCM] Make NetworkId and Junction versioned

* Fixes

---------

Co-authored-by: parity-processbot <>

* Add futures-timer

* Make cost_or_benefit public

* Update ReportPeer message format

* Add delay to reputation updates (dirtywork)

* Update ReputationAggregator

* Update tests

* Fix flucky tests

* Move reputation to state

* Use the main loop for handling reputation sendings

* Update

* Move reputation to utils

* Update reputation sending

* Fix arguments order

* Update state

* Remove new from state

* Add constant

* Add failing test for delay

* Change mocking approach

* Fix type errors

* Fix comments

* Add message handling to select

* Fix bitfields-distribution tests

* Add docs to reputation aggregator

* Replace .into_base_rep

* Use one REPUTATION_CHANGE_INTERVAL by default

* Add reputation change to statement-distribution

* Update polkadot-availability-bitfield-distribution

* Update futures selecting in subsystems

* Update reputation adding

* Send malicious changes right away without adding to state

* Add reputation to StatementDistributionSubsystem

* Handle reputation in statement distribution

* Add delay test for polkadot-statement-distribution

* Fix collator-protocol tests before applying reputation delay

* Remove into_base_rep

* Add reputation to State

* Fix failed tests

* Add reputation delay

* Update tests

* Add batched network message for peer reporting

* Update approval-distribution tests

* Update bitfield-distribution tests

* Update statement-distribution tests

* Update collator-protocol tests

* Remove levels in matching

* Address clippy errors

* Fix overseer test

* Add a metric for original count of rep changes

* Update Reputation

* Revert "Add a metric for original count of rep changes"

This reverts commit 6c9b0c1ec34491d16e562bdcba8db6b9dcf484db.

* Update node/subsystem-util/src/reputation.rs

Co-authored-by: Vsevolod Stakhov <[email protected]>

* Remove redundant vec

---------

Co-authored-by: Vsevolod Stakhov <[email protected]>

* [frame/im-online] remove external_addresses from heartbeat

* new benchmarks

* format code

* Revert "format code"

This reverts commit 586ddaa6f588b4aef92a0169cfd6a9610d1337b8.

* remove MaxPeerDataEncodingSize

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

closes paritytech/ci_cd#811

Signed-off-by: Andrei Sandu <[email protected]>

* CI job hotfix
* Instruct `cargo` to build all the binaries

* Remove redundant enter call.

* Remove optionality in dispute signature checking

* Make enter_inner and create_inherent the same.

* Remove redundant metric.

* Unification: enter and create_inherent.

* Remove `enter_inner` function.

* Remove dead code.

* Remove redundant import.

* Remove dead code in disputes.

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent

* ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent

* ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent

* Merge fix.

* Fix tests.

* Remove obsolete comment.

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent

* ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent

* ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent

* Review remarks, fixes.

* Guide updates.

* Fmt.

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent

---------

Co-authored-by: eskimor <[email protected]>
Co-authored-by: command-bot <>

* sync workspace version with 0.9.43

* sync spec and tx ver with 0.9.43

* 0943 migrations

* polkadot: update weights

* westend: update weights

* kusama: update weights

* rococo: update weights

* Move vstaging to production (and thus past session slashing).

WIP: test-runtime still needs to be fixed.

* Fix test-runtime.

---------

Co-authored-by: eskimor <[email protected]>

* para scheduler: fix  division

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update runtime/parachains/src/scheduler.rs

Co-authored-by: Liam Aharon <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

* av-store: Move prune on a separate thread

There are situations where pruning of the data could take more than a few
seconds and that might make the whole subsystem unreponsive. To avoid this just
move the prune process on a separate thread.

See: https://github.com/paritytech/polkadot/issues/7237, for more details.

Signed-off-by: Alexandru Gheorghe <[email protected]>

* av-store: Add log that prunning started

Signed-off-by: Alexandru Gheorghe <[email protected]>

* av-store: modify log severity

Signed-off-by: Alexandru Gheorghe <[email protected]>

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

* metrics: Increase the resolution of histogram metrics

These metrics are using the default histogram buckets:
```
pub const DEFAULT_BUCKETS: &[f64; 11] = &[
    0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0,
];

```

Which give us a resolution of 5ms, that's good, but there are some subsystems
where we process hundreds or even a few thousands of messages per second like
approval-voting or approval-distribution, so it makes sense to increse the
resoution of the bucket to better understand if the procesisng is in the range
of useconds.

The new bucket ranges will be:
```
[0.0001, 0.0004, 0.0016, 0.0064, 0.0256, 0.1024, 0.4096, 1.6384, 6.5536]
```

Signed-off-by: Alexandru Gheorghe <[email protected]>

* Use buckets with higher resolution

Signed-off-by: Alexandru Gheorghe <[email protected]>

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

Okay this was stupid 🤦



Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* approval-distribution: Add approvals/assignments spans on all paths

The approval and assignment logic gets called from multiple paths, so make sure
we create a tracing span on all paths to make debugging easier and be able and
correlate with the spans from approval-voting.

Signed-off-by: Alexandru Gheorghe <[email protected]>

* Tag each label with a difference tracing name

Signed-off-by: Alexandru Gheorghe <[email protected]>

* Address review feedback

Use the source to determine the tag name

Signed-off-by: Alexandru Gheorghe <[email protected]>

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* runtime/vstaging: unapplied_slashes runtime API

* runtime/vstaging: key_ownership_proof runtime API

* runtime/ParachainHost: submit_report_dispute_lost

* fix key_ownership_proof API

* runtime: submit_report_dispute_lost runtime API

* nits

* Update node/subsystem-types/src/messages.rs

Co-authored-by: Marcin S. <[email protected]>

* revert unrelated fmt changes

* dispute-coordinator: past session dispute slashing

* encapsule runtime api call for submitting report

* prettify: extract into a function

* do not exit on runtime api error

* fix tests

* try initial zombienet test

* try something

* fix a typo

* try cumulus-based collator

* fix clippy

* build polkadot-debug images with fast-runtime enabled

* wip

* runtime/inclusion: fix availability_threshold

* fix wip

* fix wip II

* revert native provider

* propagate tx submission error

* DEBUG: sync=trace

* print key ownership proof len

* panic repro

* log validator index in panic message

* post merge fixes

* replace debug assertion with a log

* fix compilation

* Let's log the dispatch info in validate block.

* fix double encoding

* Revert "Let's log the dispatch info in validate block."

This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.

* Revert "Let's log the dispatch info in validate block."

This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.

* fix compilation

* update to latest zombienet and fix test

* lower finality lag to 11

* bump zombienet again

* add a workaround, but still does not work

* Update .gitlab-ci.yml

bump zombienet.

* add a comment and search logs on all nodes

---------

Co-authored-by: Marcin S. <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Javier Viola <[email protected]>

* Implement AliasOrigin processing in XCVM

* add builder types and first test

* switch to more general builder types

* clone target for RemovePrefixAccountId32

* change builder types

* change AliasForeignAccountId32 and add test for AliasCase

* add Aliasers type to xcm configs

* add benchmark

* benchmark fix

* add benchmark function for runtimes

* fix alias_origin result types

* fix benchmark test

* add runtime-benchmarks feature in pallet-xcm-benchmarks

* fmt

* remove AliasCase, add test and fmt

* address feedback

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::generic

* ".git/.scripts/commands/bench/bench.sh" xcm westend pallet_xcm_benchmarks::generic

* ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::generic

* address feedback

* lock

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::generic

* ".git/.scripts/commands/bench/bench.sh" xcm westend pallet_xcm_benchmarks::generic

* ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::generic

* change doc

* fmt

---------

Co-authored-by: Just van Stam <[email protected]>
Co-authored-by: Just van Stam <[email protected]>
Co-authored-by: command-bot <>

* Introduce an extensible location-to-hash-account

* Convert becomes RevFallRefConvert

* Use ConvertLocation trait

* Remove Convert usage

* Builds

* Fix warnings

* Remove unused types

* Bump lock

* No need for aliasing

* Remove unused

* Deprecate legacy conversion

* Fixes

* Fixes

* Update Cargo.toml

Co-authored-by: Muharem Ismailov <[email protected]>

* Update xcm/xcm-builder/src/location_conversion.rs

Co-authored-by: Liam Aharon <[email protected]>

* Update xcm/xcm-builder/src/location_conversion.rs

Co-authored-by: Liam Aharon <[email protected]>

* Update xcm/xcm-builder/src/location_conversion.rs

Co-authored-by: Liam Aharon <[email protected]>

* Update xcm/xcm-builder/src/location_conversion.rs

Co-authored-by: Liam Aharon <[email protected]>

* Update xcm/xcm-builder/src/location_conversion.rs

Co-authored-by: Liam Aharon <[email protected]>

* Update xcm/xcm-builder/src/location_conversion.rs

Co-authored-by: Liam Aharon <[email protected]>

---------

Co-authored-by: Muharem Ismailov <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

* PVF: Instantiate wasm in pre-checking

* Move `runtime_construction_check` to prepare thread, use bytes

* [minor] Update comment

* Fix compile error

* Update Cargo.lock

* Update docs

* Add some missing docs!

* Fix XCM benchmark template

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Keep function pub

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::generic

* ".git/.scripts/commands/bench/bench.sh" xcm westend pallet_xcm_benchmarks::generic

* ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::generic

* Fix test

* Actually fix test

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

* XCM: Utility struct for AccountId conversion

* Update conversion.rs

* ParaId: Serialize/Deserialize trait implemented in no-std

* Serialize/Deserialize trait implemented in no-std for more types

* serde in deps enabled

* fix

* missing types added

* update lockfile for {"substrate"}

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: parity-processbot <>

pallet-mmr can be configured with a different hashing scheme
than frame-system pallet. Make sure to use this custom type
for MmrApi.

Signed-off-by: Adrian Catangiu <[email protected]>

* bump serde to 1.0.163

* bump ci

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* Update Cargo.lock

Signed-off-by: Alexandru Vasile <[email protected]>

* Update Cargo.lock

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: parity-processbot <>

* Move XCM query functionality to trait

* Fix tests

* Add PayOverXcm implementation

* fix the PayOverXcm trait to compile

* moved doc comment out of trait implmeentation and to the trait

* PayOverXCM documentation

* Change documentation a bit

* Added empty benchmark methods implementation and changed docs

* update PayOverXCM to convert AccountIds to MultiLocations

* Implement benchmarking method

* Change v3 to latest

* Descend origin to an asset sender (#6970)

* descend origin to an asset sender

* sender as tuple of dest and sender

* Add more variants to the QueryResponseStatus enum

* Change Beneficiary to Into<[u8; 32]>

* update PayOverXcm to return concrete errors and use AccountId as sender

* use polkadot-primitives for AccountId

* fix dependency to use polkadot-core-primitives

* force Unpaid instruction to the top of the instructions list

* modify report_outcome to accept interior argument

* use new_query directly for building final xcm query, instead of report_outcome

* fix usage of new_query to use the XcmQueryHandler

* fix usage of new_query to use the XcmQueryHandler

* tiny method calling fix

* xcm query handler (#7198)

* drop redundant query status

* rename ReportQueryStatus to OuterQueryStatus

* revert rename of QueryResponseStatus

* update mapping

* Update xcm/xcm-builder/src/pay.rs

Co-authored-by: Gavin Wood <[email protected]>

* Updates

* Docs

* Fix benchmarking stuff

* Destination can be determined based on asset_kind

* Tweaking API to minimise clones

* Some repotting and docs

---------

Co-authored-by: Anthony Alaribe <[email protected]>
Co-authored-by: Muharem Ismailov <[email protected]>
Co-authored-by: Anthony Alaribe <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

* pallet-para-config: Remove remnant WeightInfo functions

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* set_config_with_weight begone

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::configuration

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

* Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit

* Fix tests in import

* Fix the rest of the tests

* Remove dead code

* Fix todos

* Simplify session caching

* Comments for `SessionInfoProvider`

* Separate `SessionInfoProvider` from `State`

* `cache_session_info_for_head` becomes freestanding function

* Remove unneeded `mut` usage

* fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider`

* Fix SessionInfo retrieval

* Code cleanup

* Don't wrap `SessionInfoProvider` in an `Option`

* Remove `earliest_session()`

* Remove pre-caching -> wip

* Fix some tests and code cleanup

* Fix all tests

* Fixes in tests

* Fix comments, variable names and small style changes

* Fix a warning

* impl From<SessionWindowSize> for NonZeroUsize

* Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG

* Code review feedback

* Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db

* Remove `col_session_data` usages

* Storage migration clearing columns w/o removing them

* Remove session data column usages from `approval-voting` and `dispute-coordinator` tests

* Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests

* Fix formatting in initialized.rs

* Fix a corner case in `SessionInfo` caching for `dispute-coordinator`

* Remove `RollingSessionWindow` ;(

* Revert "Fix formatting in initialized.rs"

This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc.

* v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it

* Fix `NUM_COLUMNS` in `approval-voting`

* Use `columns::v3::NUM_COLUMNS` when opening db

* Update node/service/src/parachains_db/upgrade.rs

Co-authored-by: Andrei Sandu <[email protected]>

* Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3`

* Fix `NUM+COLUMNS` in approval_voting

* Fix formatting

* Fix columns usage

* Clarification comments about the different db versions

---------

Co-authored-by: Andrei Sandu <[email protected]>

* bounded-collections bump to 0.1.7

Companion for: paritytech/substrate#14225

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* To revert: Update substrate branch to "lexnv/bump_sp_crates"

Signed-off-by: Alexandru Vasile <[email protected]>

* Revert "To revert: Update substrate branch to "lexnv/bump_sp_crates""

This reverts commit 5f1db84eac4a226c37b7f6ce6ee19b49dc7e2008.

* Update cargo lock

Signed-off-by: Alexandru Vasile <[email protected]>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <[email protected]>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>

* Companion for Substrate#14228

https://github.com/paritytech/substrate/pull/14228

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* XCM: Fix issue with RequestUnlock

* Leave API changes for v4

* Fix clippy errors

* Fix tests

---------

Co-authored-by: parity-processbot <>

* Document usage of gum crate

* Small fix

* Add some more basic info

* Update node/gum/src/lib.rs

Co-authored-by: Andrei Sandu <[email protected]>

* Update target docs

---------

Co-authored-by: Andrei Sandu <[email protected]>

* runtime/vstaging: unapplied_slashes runtime API

* runtime/vstaging: key_ownership_proof runtime API

* runtime/ParachainHost: submit_report_dispute_lost

* fix key_ownership_proof API

* runtime: submit_report_dispute_lost runtime API

* nits

* Update node/subsystem-types/src/messages.rs

Co-authored-by: Marcin S. <[email protected]>

* revert unrelated fmt changes

* post merge fixes

* fix compilation

---------

Co-authored-by: Marcin S. <[email protected]>

We consider BEEFY mature enough to run by default on all nodes
for test networks (Rococo/Wococo/Versi).

Right now, most nodes are not running it since it's opt-in using
--beefy flag. Switch to an opt-out model for test networks.

Replace --beefy flag from CLI with --no-beefy and have BEEFY
client start by default on test networks.

Signed-off-by: acatangiu <[email protected]>