- Dec 13, 2023
-
-
Squirrel authored
We currently use a bit of a hack in `.cargo/config` to make sure that clippy isn't too annoying by specifying the list of lints. There is now a stable way to define lints for a workspace. The only down side is that every crate seems to have to opt into this so there's a *few* files modified in this PR. Dependencies: - [x] PR that upgrades CI to use rust 1.74 is merged. --------- Co-authored-by: joe petrowski <[email protected]> Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: Liam Aharon <[email protected]>
-
Alexandru Gheorghe authored
Initial implementation for the plan discussed here: https://github.com/paritytech/polkadot-sdk/issues/701 Built on top of https://github.com/paritytech/polkadot-sdk/pull/1178 v0: https://github.com/paritytech/polkadot/pull/7554, ## Overall idea When approval-voting checks a candidate and is ready to advertise the approval, defer it in a per-relay chain block until we either have MAX_APPROVAL_COALESCE_COUNT candidates to sign or a candidate has stayed MAX_APPROVALS_COALESCE_TICKS in the queue, in both cases we sign what candidates we have available. This should allow us to reduce the number of approvals messages we have to create/send/verify. The parameters are configurable, so we should find some values that balance: - Security of the network: Delaying broadcasting of an approval shouldn't but the finality at risk and to make sure that never happens we won't delay sending a vote if we are past 2/3 from the no-show time. - Scalability of the network: MAX_APPROVAL_COALESCE_COUNT = 1 & MAX_APPROVALS_COALESCE_TICKS =0, is what we have now and we know from the measurements we did on versi, it bottlenecks approval-distribution/approval-voting when increase significantly the number of validators and parachains - Block storage: In case of disputes we have to import this votes on chain and that increase the necessary storage with MAX_APPROVAL_COALESCE_COUNT * CandidateHash per vote. Given that disputes are not the normal way of the network functioning and we will limit MAX_APPROVAL_COALESCE_COUNT in the single digits numbers, this should be good enough. Alternatively, we could try to create a better way to store this on-chain through indirection, if that's needed. ## Other fixes: - Fixed the fact that we were sending random assignments to non-validators, that was wrong because those won't do anything with it and they won't gossip it either because they do not have a grid topology set, so we would waste the random assignments. - Added metrics to be able to debug potential no-shows and mis-processing of approvals/assignments. ## TODO: - [x] Get feedback, that this is moving in the right direction. @ordian @sandreim @eskimor @burdges, let me know what you think. - [x] More and more testing. - [x] Test in versi. - [x] Make MAX_APPROVAL_COALESCE_COUNT & MAX_APPROVAL_COALESCE_WAIT_MILLIS a parachain host configuration. - [x] Make sure the backwards compatibility works correctly - [x] Make sure this direction is compatible with other streams of work: https://github.com/paritytech/polkadot-sdk/issues/635 & https://github.com/paritytech/polkadot-sdk/issues/742 - [x] Final versi burn-in before merging --------- Signed-off-by: Alexandru Gheorghe <[email protected]>
-
- Dec 12, 2023
-
-
Branislav Kontur authored
## Summary This PR turns on `-D warnings` for `cargo-check-each-crate job` job to fail on warnings e.g. like this: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4673130 Before this PR, there was a warning and `cargo-check-each-crate` job did not fail: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4641444 ``` warning: unused import: `ToTokens` --> substrate/primitives/api/proc-macro/src/utils.rs:22:34 | 22 | use quote::{format_ident, quote, ToTokens}; | ^^^^^^^^ | = note: `#[warn(unused_imports)]` on by default warning: `sp-api-proc-macro` (lib) generated 1 warning (run `cargo fix --lib -p sp-api-proc-macro` to apply 1 suggestion) ``` Fixes on the way: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4641444 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4673265 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4673410 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4673681 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4673836 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4673941 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4674256 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4679328 ## Questions - [ ] why does this check triggers only `cargo check --locked`? `--all-features` or `--all-targets` are not needed? Or aren't they avoided intentionally? --------- Co-authored-by: command-bot <>
-
- Dec 06, 2023
-
-
Alexander Samusev authored
-
- Dec 05, 2023
-
-
Marcin S. authored
Co-authored-by: Javier Viola <[email protected]>
-
juangirini authored
This PR is a continuation of https://github.com/paritytech/polkadot-sdk/pull/2102 and part of an initiative started here https://hackmd.io/@romanp/rJ318ZCEp What has been done: - The content under `docs/*` (with the exception of `docs/mermaid`) has been moved to `docs/contributor/` - Developer Hub has been renamed to Polkadot SDK Docs, and the crate has been renamed from `developer-hub` to `polkadot-sdk-docs` - The content under `developer-hub/*` has been moved to `docs/sdk` --- Original PR https://github.com/paritytech/polkadot-sdk/pull/2565 , it has been close due to too many rebase conflicts --------- Co-authored-by: Serban Iorga <[email protected]> Co-authored-by: Chevdor <[email protected]> Co-authored-by: Egor_P <[email protected]> Co-authored-by: Bastian Köcher <[email protected]>
-
Vladimir Istyufeev authored
-
- Dec 01, 2023
-
-
Liam Aharon authored
Using taplo, fixes all our broken and inconsistent toml formatting and adds CI to keep them tidy. If people want we can customise the format rules as described here https://taplo.tamasfe.dev/configuration/formatter-options.html @ggwpez , I suggest zepter is used only for checking features are propagated, and leave formatting for taplo to avoid duplicate work and conflicts. TODO - [x] Use `exclude = [...]` syntax in taplo file to ignore zombienet tests instead of deleting the dir --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Bastian Köcher <[email protected]>
-
- Nov 30, 2023
-
-
Kian Paimani authored
This PR introduces the new crate `developer_hub` into the polkadot-sdk repo. The vision for the developer-hub crate is detailed in [this document](https://docs.google.com/document/d/1XLLkFNE8v8HLvZpI2rzsa8N2IN1FcKntc8q-Sc4xBAk/edit?usp=sharing). <img width="1128" alt="Screenshot 2023-11-02 at 10 45 48" src="https://github.com/paritytech/polkadot-sdk/assets/5588131/1e12b60f-fef5-42c4-8503-a3ba234077c3"> Other than adding the new crate, it also does the following: * Remove the `substrate` crate, as there is now a unique umbrella crate for multiple things in `developer_hub::polkadot_sdk`. * (backport candidate) A minor change to `frame-support` macros that allows `T::RuntimeOrigin` to also be acceptable as the origin type. * (backport candidate) A minor change to `frame-system` that allows us to deposit events at genesis because now the real genesis config is generated via wasm, and we can safely assume `cfg!(feature = "std")` means only testing. related to #62. * (backport candidate) Introduces a small `read_events_for_pallet` to `frame_system` for easier event reading in tests. * From https://github.com/paritytech/polkadot-sdk-docs/issues/31, it takes action on improving the `pallet::call` docs. * From https://github.com/paritytech/polkadot-sdk-docs/issues/31, it takes action on improving the `UncheckedExtrinsic` docs. ## Way Forward First, a version of this is deployed temporarily [here](https://blog.kianenigma.nl/polkadot-sdk/developer_hub/index.html). I will keep it up to date on a daily basis. ### This Pull Request I see two ways forward: 1. We acknowledge that everything in `developer-hub` is going to be WIP, and merge this asap. We should not yet use links to this crate anywhere. 2. We make this be the feature branch, make PRs against this, and either gradually backport it, or only merge to master once it is done. I am personally in favor of option 1. If we stick to option 2, we need a better way to deploy a staging version of this to gh-pages. ### Issue Tracking The main issues related to the future of `developer_hub` are: - https://github.com/paritytech/polkadot-sdk-docs/issues/31 - https://github.com/paritytech/polkadot-sdk-docs/issues/4 - https://github.com/paritytech/polkadot-sdk-docs/issues/26 - https://github.com/paritytech/polkadot-sdk-docs/issues/32 - https://github.com/paritytech/polkadot-sdk-docs/issues/36 ### After This Pull Request - [ ] create a redirect for https://paritytech.github.io/polkadot-sdk/master/substrate/ - [x] analytics - [ ] link checker - [ ] the matter of publishing, and how all of these relative links for when we do, that is still an open question. There is section on this in the landing page. - [ ] updated https://paritytech.github.io/ --------- Co-authored-by: Liam Aharon <[email protected]> Co-authored-by: Juan Girini <[email protected]> Co-authored-by: bader y <[email protected]> Co-authored-by: Sebastian Kunert <[email protected]> Co-authored-by: James Wilson <[email protected]> Co-authored-by: Michal Kucharczyk <[email protected]>
-
- Nov 29, 2023
-
-
Vladimir Istyufeev authored
-
Dónal Murray authored
Since the Polkadot and Kusama runtimes are no longer in the repo, the relevant systems parachains runtimes also need to be removed. More context [here](https://github.com/paritytech/polkadot-sdk/issues/603) and [here](https://github.com/paritytech/polkadot-sdk/pull/1731). Removes the following: - `asset-hub-kusama` and `asset-hub-polkadot` - `bridge-hub-kusama` and `bridge-hub-polkadot` - `collectives-polkadot` - `glutton-kusama` Partially solves #603 and adds to #1731.
-
- Nov 28, 2023
-
-
s0me0ne-unkn0wn authored
Co-authored-by: ordian <[email protected]> Co-authored-by: Vladimir Istyufeev <[email protected]>
-
Vladimir Istyufeev authored
-
- Nov 27, 2023
-
-
Maciej authored
**Overview:** Adding an extra malus variant focusing on disputing finalized blocks. It will: - wrap around approval-voting - listen to `OverseerSignal::BlockFinalized` and when encountered start a dispute for the `dispute_offset`th ancestor - simply pass through all other messages and signals Add zombienet tests testing various edgecases: - disputing freshly finalized blocks - disputing stale finalized blocks - disputing eagerly pruned finalized blocks (might be separate PR) **TODO:** - [x] Register new malus variant - [x] Simple pass through wrapper (approval-voting) - [x] Simple network definition - [x] Listen to block finalizations - [x] Fetch ancestor hash - [x] Fetch session index - [x] Fetch candidate - [x] Construct and send dispute message - [x] zndsl test 1 checking that disputes on fresh finalizations resolve valid Closes #1365 - [x] zndsl test 2 checking that disputes for too old finalized blocks are not possible Closes #1364 - [ ] zndsl test 3 checking that disputes for candidates with eagerly pruned relay parent state are handled correctly #1359 (deferred to a separate PR) - [x] Unit tests for new malus variant (testing cli etc) - [x] Clean/streamline error handling - [ ] ~~Ensure it tests properly on session boundaries~~ --------- Co-authored-by: Javier Viola <[email protected]> Co-authored-by: Marcin S. <[email protected]> Co-authored-by: Tsvetomir Dimitrov <[email protected]>
-
- Nov 25, 2023
-
-
Marcin S. authored
-
- Nov 24, 2023
-
-
Javier Viola authored
Add `build-push-image-polkadot-debug` job to needs since we changed to use the polkadot-debug image from the current branch for cumulus test and we need to be sure that the image is ready. Fix issues like https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4481059 cc: @bkchr Co-authored-by: Bastian Köcher <[email protected]>
-
Javier Viola authored
Include fix for `0002-parachains-upgrade-smoke-test` test.
-
- Nov 23, 2023
-
-
Liam Aharon authored
CI: Disable runtime upgrade spec name check on Westend Asset Hub and fix Staking pallet migration (#2447) Westend Asset Hub currently failing because the spec name is being changed next runtime upgrade (https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4413125). This also fixes an idempotency issue with a staking pallet migration. Similar issues will be caught automatically now that we've also updated to try-runtime-cli v0.5.0 which checks for idempotency issues. This also enables try-state checks running in the CI.
-
- Nov 22, 2023
-
-
Michal Kucharczyk authored
The out-dated version (bad tag) of [polkadot image](https://github.com/paritytech/polkadot-sdk/blob/ede4a362622dfa69eb167eaa876246b1289f4b41/.gitlab/pipeline/zombienet/cumulus.yml#L31) ([docker info](https://hub.docker.com/layers/paritypr/polkadot-debug/master/images/sha256:adb1658052cf671b50c90d5cece5c7a131efa1a95978249bd5cb85a5ad654f7a?context=explore)) was used. This PR fixes this. See also: https://github.com/paritytech/polkadot-sdk/pull/2411#issuecomment-1822632724 Also adds an abstraction that allows asynchronous backends to be passed to `ChainApiSubsystem` --------- Co-authored-by: Sebastian Kunert <[email protected]>
-
Marcin S. authored
-
Alexander Samusev authored
Docker images from paritypr are also used in testnets. PR reverts docker tags naming to a more convenient. cc @PierreBesson
-
- Nov 21, 2023
-
-
Javier Viola authored
Includes: - bump `pjs` modules versions - re-enable test disable in https://github.com/paritytech/polkadot-sdk/pull/2294 --------- Co-authored-by: Bastian Köcher <[email protected]>
-
- Nov 17, 2023
-
-
Liam Aharon authored
Westend Collectives migration CI check can be fixed once we have https://github.com/paritytech/try-runtime-cli/pull/58, will open another PR once it is available. - [x] Remove deprecated `DmpQueue` pallet from Rococo Contracts, the migration is complete - [x] Fix Asset Hub Rococo storage versions - [x] Add migration check CI for Asset Hub Rococo and Westend Bridge Hub
-
- Nov 16, 2023
-
-
Alexander Samusev authored
Since preparation for the merge queues needs more time I'm enabling zombienet jobs in PRs CI back.
-
- Nov 15, 2023
-
-
Dónal Murray authored
Add collectives and glutton parachain westend runtimes to prepare for #1737. The removal of system parachain native runtimes #1737 is blocked until chainspecs and runtime APIs can be dealt with cleanly (merge of #1256 and follow up PRs). In the meantime, these additions are ready to be merged to `master`, so I have separated them out into this PR. Also marked `bridge-hub-westend` as unimplemented in line with [this issue](https://github.com/paritytech/parity-bridges-common/issues/2602 ). TODO - [x] add to `command-bot` benchmarks - [x] add to `command-bot-scripts` benchmarks - [x] generate weights --------- Co-authored-by: joe petrowski <[email protected]> Co-authored-by: Muharem <[email protected]> Co-authored-by: command-bot <> Co-authored-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Branislav Kontur <[email protected]>
-
Alexander Samusev authored
PR prepares CI to the GitHub Merge Queues. All github actions that were running in PR adjusted so they can run in the merge queues. Zombienet jobs will do nothing during PRs but they will run during merge queues. Jobs that will be skipped during PR: - all zombienet jobs - all publish docker jobs Jobs that will be skipped during merge queue: - check-labels - check-prdoc - pr-custom-review - review trigger cc https://github.com/paritytech/ci_cd/issues/862
-
- Nov 13, 2023
-
-
gupnik authored
-
- Nov 06, 2023
-
-
Andrei Sandu authored
**_PR migrated from https://github.com/paritytech/polkadot/pull/6782_** This PR will upgrade the network protocol to version 3 -> VStaging which will later be renamed to V3. This version introduces a new kind of assignment certificate that will be used for tranche0 assignments. Instead of issuing/importing one tranche0 assignment per candidate, there will be just one certificate per relay chain block per validator. However, we will not be sending out the new assignment certificates, yet. So everything should work exactly as before. Once the majority of the validators have been upgraded to the new protocol version we will enable the new certificates (starting at a specific relay chain block) with a new client update. There are still a few things that need to be done: - [x] Use bitfield instead of Vec<CandidateIndex>: https://github.com/paritytech/polkadot/pull/6802 - [x] Fix existing approval-distribution and approval-voting tests - [x] Fix bitfield-distribution and statement-distribution tests - [x] Fix network bridge tests - [x] Implement todos in the code - [x] Add tests to cover new code - [x] Update metrics - [x] Remove the approval distribution aggression levels: TBD PR - [x] Parachains DB migration - [x] Test network protocol upgrade on Versi - [x] Versi Load test - [x] Add Zombienet test - [x] Documentation updates - [x] Fix for sending DistributeAssignment for each candidate claimed by a v2 assignment (warning: Importing locally an already known assignment) - [x] Fix AcceptedDuplicate - [x] Fix DB migration so that we can still keep old data. - [x] Final Versi burn in --------- Signed-off-by: Andrei Sandu <[email protected]> Signed-off-by: Alexandru Gheorghe <[email protected]> Co-authored-by: Alexandru Gheorghe <[email protected]>
-
- Nov 05, 2023
-
-
Oliver Tale-Yazdi authored
Otherwise the return code is not correctly propagated (ref https://github.com/ggwpez/zepter/pull/48 ). --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]>
-
- Nov 02, 2023
-
-
Branislav Kontur authored
-
- Oct 31, 2023
-
-
Lulu authored
Co-authored-by: Sergejs Kostjucenko <[email protected]> Co-authored-by: Bastian Köcher <[email protected]>
-
Bastian Köcher authored
This pull request changes how `check-each-crate.py` is working. Instead of passing the name of the crate via `-p`, we now jump into the directory of the crate and call there `cargo check`. This should fix issues like https://github.com/paritytech/polkadot-sdk/issues/2013 where a crate is present twice in the `Cargo.lock`. Besides that it also changes `core/Cargo.toml` to not always pull in bandersnatch.
-
- Oct 30, 2023
-
-
Liam Aharon authored
Runtime migration CI is currently failing (https://gitlab.parity.io/parity/mirrors/polkadot-sdk/builds/4122083) for the contracts testnet due to unnecessary panicing in a `pre_upgrade` hook. Soon idempotency will be enforced https://github.com/paritytech/try-runtime-cli/issues/42, in the mean time we need to manually fix these issues as they arise. --- also removes backticks from the string in `echo`, which caused a 'command not found' error in ci output
-
- Oct 24, 2023
-
-
Alexander Samusev authored
Need to run this job in master to make it `Required`
-
Oliver Tale-Yazdi authored
Adds a config file that allows to run `zepter` without any arguments in the workspace to address all issues. A secondary workflow for the CI is provided as `zepter run check`. Both the formatting and linting are now in one check for efficiancy. The latest version also detects some more things that `featalign` was already showing. Error message [in the CI](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3916205) now looks like this: ```pre ... crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml) feature 'std' must propagate to: parity-scale-codec Found 55 issues (run with --fix to fix). Error: Command 'lint propagate-feature' failed with exit code 1 Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration. It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter...
-
- Oct 19, 2023
-
-
Liam Aharon authored
`cargo install` takes a long time in CI. We want to run it relatively frequently without chewing through so much compute (see https://github.com/paritytech/ci_cd/issues/771) so I added automatic binary releases to the try-runtime-cli repo. A small added benefit is we can use it in our existing `on-runtime-upgrade` checks, which should cut their execution time by about half.
-
- Oct 18, 2023
-
-
Adrian Catangiu authored
cumulus: add asset-hub-rococo runtime based on asset-hub-kusama and add asset-bridging support to it (#1215) This commit adds Rococo Asset Hub dedicated runtime so we can test new features here, before merging them in Kusama Asset Hub. Also adds one such feature: asset transfer over bridge (Rococo AssetHub <> Wococo AssetHub) - clone `asset-hub-kusama-runtime` -> `asset-hub-rococo-runtime` - make it use Rococo primitives, names, assets, constants, etc - add asset-transfer-over-bridge support to Rococo AssetHub <> Wococo AssetHub Fixes #1128 --------- Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: joe petrowski <[email protected]> Co-authored-by: Francisco Aguirre <[email protected]>
-
- Oct 17, 2023
-
-
Marcin S. authored
Co-authored-by: Mira Ressel <[email protected]>
-
Mira Ressel authored
These two jobs are now required by github branch protection settings.
-
- Oct 11, 2023
-
-
Mira Ressel authored
Co-authored-by: command-bot <>
-