* wait for relay chain to sync then get parachain header

* Spawn new thread to wait for the target block

* second round of comments from the PR on substrate

* third round of pr comments

* add zombienet tests

* rebase issues

* refactor tests based on pr comments

* rebase issues

* pr comments

* passing zombienet test

* cargo +nightly fmt

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* use cargo lock from master

* pr comments

* cargo fmt

* use finalised block instead of best block

* use import notification stream

* rebase changes

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/relay-chain-interface/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/relay-chain-interface/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* pr comments

* use new file names

* db snaphots moved to google cloud storage

* Update client/network/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/lib.rs

Co-authored-by: Sebastian Kunert <[email protected]>

* pr comments

* Update zombienet/tests/0007-full_node_warp_sync.toml

Co-authored-by: Sebastian Kunert <[email protected]>

* Update zombienet/tests/0007-full_node_warp_sync.toml

Co-authored-by: Sebastian Kunert <[email protected]>

* Scenario 1

Parachain node and in-node relay chain both start with --sync warp. This ensures that the waiting logic works as expected.

Scenario 2
Parachain node starts with warp sync, relay chain points to a node already synced up

scenario 3
Parachain node starts with warp sync, relay chain points to a node that uses warp sync

* Use test-parachain

* use test-parachain chainspecs

* remove relay chain spec as it is no longer required

* add back relaychain spec file

* pr comments

* Upload snapshots to google cloud

* Update zombienet/tests/0007-prepare-warp-sync-db-snapshot.md

Co-authored-by: Sebastian Kunert <[email protected]>

* update documentation

* Fix snapshot URLs

* use master lock file

* add finalized_block_hash

* Patch diener for CI

* Bump Zombienet

* Add 0007 zombienet test

* Bump zombienet

* Revert "Patch diener for CI"

This reverts commit 9ece6c9fc9b17058b61cd7e9dee29d3a9af87841.

* merge fixes

* use master lock file

* Update Substrate & Polkadot

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Michal Kucharczyk <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/paritytech/parity-scale-codec/releases)
- [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/parity-scale-codec/compare/parity-scale-codec-3.3.0...v3.4.0

)

---
updated-dependencies:
- dependency-name: parity-scale-codec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Increase delay for pov-recovery

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Comment

* FMT

* Clear waiting_recovery when block is recovered or recovery failed

* Introduce recovery queue that preserved insertion order

* Better error logs

* Decrease slot duration

* Style improvements

* Add option to use unordered queue

* Maintain cache of finalized blocks

* Wait for one relay chain slot before recovery

* Make retries testable

* fmt

* Improve docs

* Improve docs

* Simplify RecoveryQueue

* Remove unwanted changes

* Adjust to comments

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Move recovery delay into the queue

* Check for finalized number

* Clean up

* Use timer

Co-authored-by: Bastian Köcher <[email protected]>

* Simplify implementation

* Revert "Use timer"

This reverts commit 3809eed840d3a09d54212f99486782ff80cdc1c9.

* Properly clear `to_recover` flag

---------

Co-authored-by: Bastian Köcher <[email protected]>

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.91 to 1.0.92.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.91...v1.0.92

)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0

)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec) from 3.2.2 to 3.3.0.
- [Release notes](https://github.com/paritytech/parity-scale-codec/releases)
- [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/parity-scale-codec/compare/parity-scale-codec-v3.2.2...parity-scale-codec-3.3.0

)

---
updated-dependencies:
- dependency-name: parity-scale-codec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [clap](https://github.com/clap-rs/clap) from 4.1.1 to 4.1.4.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.1.1...v4.1.4

)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Skip verification on state import

* Actually use existing variables

* Merge checks

* Remove assumption that there is a local bootnode

* Bump zombienet to 1.3.29

* Separate "--"

Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.60 to 0.1.63.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.60...0.1.63

)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.1 to 1.24.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits

)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump lru from 0.8.1 to 0.9.0

Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.8.1 to 0.9.0.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.8.1...0.9.0

)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update substrate and polkadot

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Keith Yeung <[email protected]>

Bumps [clap](https://github.com/clap-rs/clap) from 4.0.32 to 4.1.1.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.32...v4.1.1

)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

The runtime function name doesn't change on chain, especially not for old runtimes.

* Use primitives reexported from `polkadot_primitives` crate root

* restart CI

* Fixes after merge

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.23.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.23.0...tokio-1.24.1

)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove the `RelaychainClient` trait

It was just some historical trait that isn't really required anymore. Besides that this pr re-exports
types that are being used by the relay chain interface to make its usage easier.

* Fix warning

* BlockId removal: refactor: BlockBackend::block|block_status

It changes the arguments of:
-  `BlockBackend::block`
-  `BlockBackend::block_status`

method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.151 to 1.0.152.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.151...v1.0.152

)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.59 to 0.1.60.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.59...0.1.60

)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [clap](https://github.com/clap-rs/clap) from 4.0.29 to 4.0.32.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.29...v4.0.32

)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.89 to 1.0.91.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.89...v1.0.91

)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* BlockId removal: refactor: HeaderBackend::status

It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>

* BlockId removal: refactor: HeaderBackend::header

It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`

methods from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>

Prevents the StateDbError::TooManySiblingBlocks error from being triggered by eagerly removing 
stale blocks from the backend on block import and before the error condition is met.

Introduces a just in time block recovery mechanism for blocks that were wrongly removed
via an explicit pov-recovery method

Co-authored-by: Bastian Köcher <[email protected]>

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.37...1.0.38

)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [dyn-clone](https://github.com/dtolnay/dyn-clone) from 1.0.9 to 1.0.10.
- [Release notes](https://github.com/dtolnay/dyn-clone/releases)
- [Commits](https://github.com/dtolnay/dyn-clone/compare/1.0.9...1.0.10

)

---
updated-dependencies:
- dependency-name: dyn-clone
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.150 to 1.0.151.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.150...v1.0.151

)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.149 to 1.0.150.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.149...v1.0.150

)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Allow specification of multiple urls for relay chain rpc nodes

* Add pooled RPC client basics

* Add list of clients to pooled client

* Improve

* Forward requests to dispatcher

* Switch clients on error

* Implement rotation logic

* Improve subscription handling

* Error handling cleanup

* Remove retry from rpc-client

* Improve naming

* Improve documentation

* Improve `ClientManager` abstraction

* Adjust zombienet test

* Add more comments

* fmt

* Apply reviewers comments

* Extract reconnection to extra method

* Add comment to reconnection method

* Clean up some dependencies

* Fix build

* fmt

* Provide alias for cli argument

* Apply review comments

* Rename P* to Relay*

* Improve zombienet test

* fmt

* Fix zombienet sleep

* Simplify zombienet test

* Reduce log clutter and fix starting position

* Do not distribute duplicated imported and finalized blocks

* fmt

* Apply code review suggestions

* Move building of relay chain interface to `cumulus-client-service`

* Refactoring to not push back into channel

* FMT

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.22.0...tokio-1.23.0

)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* companion for #12663 jsonrpsee v0.16.1

* update deps

* update rpc client params

* update substrate and polkadot

* Companion for paritytech/substrate#12764

* Remove `async-trait`

* Fix trait

* update lockfile for {"substrate", "polkadot"}

Co-authored-by: parity-processbot <>

* Make tokio executor mandatory

* kick CI

* Update Cargo.lock

* Provisioner companion changes

* update substrate commit hash

* Update Substrate & Polkadot

Co-authored-by: Bastian Köcher <[email protected]>

Bumps [array-bytes](https://github.com/hack-ink/array-bytes) from 4.2.0 to 6.0.0.
- [Release notes](https://github.com/hack-ink/array-bytes/releases)
- [Changelog](https://github.com/hack-ink/array-bytes/blob/main/CHANGELOG)
- [Commits](https://github.com/hack-ink/array-bytes/compare/v4.2.0...v6.0.0

)

---
updated-dependencies:
- dependency-name: array-bytes
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.58 to 0.1.59.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.58...0.1.59

)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>