1. Jan 10, 2024
    • ordian's avatar
      statement-distribution: validator disabling (#1841) · a4195326
      ordian authored
      
      
      Closes #1591.
      
      The purpose of this PR is filter out backing statements from the network
      signed by disabled validators. This is just an optimization, since we
      will do filtering in the runtime in #1863 to avoid nodes to filter
      garbage out at block production time.
      
      - [x] Ensure it's ok to fiddle with the mask of manifests
      - [x] Write more unit tests
      - [x] Test locally
      - [x] simple zombienet test
      - [x] PRDoc
      
      ---------
      
      Co-authored-by: default avatarTsvetomir Dimitrov <[email protected]>
      a4195326
  2. Dec 13, 2023
    • Alexandru Gheorghe's avatar
      Approve multiple candidates with a single signature (#1191) · a84dd0db
      Alexandru Gheorghe authored
      Initial implementation for the plan discussed here: https://github.com/paritytech/polkadot-sdk/issues/701
      Built on top of https://github.com/paritytech/polkadot-sdk/pull/1178
      v0: https://github.com/paritytech/polkadot/pull/7554,
      
      ## Overall idea
      
      When approval-voting checks a candidate and is ready to advertise the
      approval, defer it in a per-relay chain block until we either have
      MAX_APPROVAL_COALESCE_COUNT candidates to sign or a candidate has stayed
      MAX_APPROVALS_COALESCE_TICKS in the queue, in both cases we sign what
      candidates we have available.
      
      This should allow us to reduce the number of approvals messages we have
      to create/send/verify. The parameters are configurable, so we should
      find some values that balance:
      
      - Security of the network: Delaying broadcasting of an approval
      shouldn't but the finality at risk and to make sure that never happens
      we won't delay sending a vote if we are past 2/3 from the no-show time.
      - Scalability of the network: MAX_APPROVAL_COALESCE_COUNT = 1 &
      MAX_APPROVALS_COALESCE_TICKS =0, is what we have now and we know from
      the measurements we did on versi, it bottlenecks
      approval-distribution/approval-voting when increase significantly the
      number of validators and parachains
      - Block storage: In case of disputes we have to import this votes on
      chain and that increase the necessary storage with
      MAX_APPROVAL_COALESCE_COUNT * CandidateHash per vote. Given that
      disputes are not the normal way of the network functioning and we will
      limit MAX_APPROVAL_COALESCE_COUNT in the single digits numbers, this
      should be good enough. Alternatively, we could try to create a better
      way to store this on-chain through indirection, if that's needed.
      
      ## Other fixes:
      - Fixed the fact that we were sending random assignments to
      non-validators, that was wrong because those won't do anything with it
      and they won't gossip it either because they do not have a grid topology
      set, so we would waste the random assignments.
      - Added metrics to be able to debug potential no-shows and
      mis-processing of approvals/assignments.
      
      ## TODO:
      - [x] Get feedback, that this is moving in the right direction. @ordian
      @sandreim @eskimor @burdges, let me know what you think.
      - [x] More and more testing.
      - [x]  Test in versi.
      - [x] Make MAX_APPROVAL_COALESCE_COUNT &
      MAX_APPROVAL_COALESCE_WAIT_MILLIS a parachain host configuration.
      - [x] Make sure the backwards compatibility works correctly
      - [x] Make sure this direction is compatible with other streams of work:
      https://github.com/paritytech/polkadot-sdk/issues/635 &
      https://github.com/paritytech/polkadot-sdk/issues/742
      
      
      - [x] Final versi burn-in before merging
      
      ---------
      
      Signed-off-by: default avatarAlexandru Gheorghe <[email protected]>
      a84dd0db
  3. Dec 05, 2023
  4. Nov 14, 2023
  5. Nov 06, 2023
    • Andrei Sandu's avatar
      approval-voting improvement: include all tranche0 assignments in one certificate (#1178) · 0570b6fa
      Andrei Sandu authored
      **_PR migrated from https://github.com/paritytech/polkadot/pull/6782_** 
      
      This PR will upgrade the network protocol to version 3 -> VStaging which
      will later be renamed to V3. This version introduces a new kind of
      assignment certificate that will be used for tranche0 assignments.
      Instead of issuing/importing one tranche0 assignment per candidate,
      there will be just one certificate per relay chain block per validator.
      However, we will not be sending out the new assignment certificates,
      yet. So everything should work exactly as before. Once the majority of
      the validators have been upgraded to the new protocol version we will
      enable the new certificates (starting at a specific relay chain block)
      with a new client update.
      
      There are still a few things that need to be done:
      
      - [x] Use bitfield instead of Vec<CandidateIndex>:
      https://github.com/paritytech/polkadot/pull/6802
      
      
        - [x] Fix existing approval-distribution and approval-voting tests
        - [x] Fix bitfield-distribution and statement-distribution tests
        - [x] Fix network bridge tests
        - [x] Implement todos in the code
        - [x] Add tests to cover new code
        - [x] Update metrics
        - [x] Remove the approval distribution aggression levels: TBD PR
        - [x] Parachains DB migration 
        - [x] Test network protocol upgrade on Versi
        - [x] Versi Load test
        - [x] Add Zombienet test
        - [x] Documentation updates
      - [x] Fix for sending DistributeAssignment for each candidate claimed by
      a v2 assignment (warning: Importing locally an already known assignment)
       - [x]  Fix AcceptedDuplicate
       - [x] Fix DB migration so that we can still keep old data.
       - [x] Final Versi burn in
      
      ---------
      
      Signed-off-by: default avatarAndrei Sandu <[email protected]>
      Signed-off-by: default avatarAlexandru Gheorghe <[email protected]>
      Co-authored-by: default avatarAlexandru Gheorghe <[email protected]>
      0570b6fa
  6. Oct 21, 2023
  7. Sep 27, 2023