Reduce network bandwidth, improve parablock times: optimize approval-distribution (#5164)
* gossip-support: be explicit about dimensions * some guide updates * update network-bridge to distinguish x and y dimensions * get everything to compile * beginnings * some TODOs * polkadot runtime: use relevant_authorities * make gossip topologies per-session * better formatting * gossip support: use current session validators * expand in comment * adjust tests and fix index bug * add past/present/future connection test and clean up code * fmt * network bridge: updated types * update protocols to new gossip topology message * guide updates * add session to BlockApprovalMeta * add session to block info * refactor knowledge and remove most unify logic * start replacing gossip_peers with new SessionTopologies * add routing information to message state * add some utilities to SessionTopology * implement new gossip topology logic * re-implement unify_with_peer * distribute assignments according to topology * finish grid topology implementation * refactor network bridge slightly * issue connection requests on all past/present/future * fmt * address grumbles * tighten invariants in unify_with_peer * implement random propagation * refactor: extract required routing adjustment logic * some block-age logic * aggressively propagate messages when finality is slow * overhaul aggression system to have 3 levels * add aggression metrics * remove aggression L3 * reduce random circulation * remove PeerData * get approval tests compiling * use btree_map in known_by to make deterministic * Revert "use btree_map in known_by to make deterministic" This reverts commit 330d65343a7bb6fe4dd0f24bd8dbc15c0cbdbd9d. * test XY grid propagation * remove stray println * test unshared dimension propagation * add random gossip check * test unify_with_peer better * test sending after getting gossip topology * test L1 aggression on originator * test L1 aggression for non-originators * test non-originator aggression L2 * fnt * ~spellcheck * fix statement-distribution tests * fix flaky test * fix metrics typo * re-send periodically * test resending * typo Co-authored-by:Bernhard Schuster <bernhard@ahoi.io> * add more metrics about apd messages * add back unify_with_peer logs * make Resend an enum * be more explicit when resending * fmt * fix error * add a TODO for refactoring * remove debug metrics * add some guide stuff * fmt * update runtime API in test-runtim Co-authored-by:
Bernhard Schuster <bernhard@ahoi.io>
Showing
- polkadot/Cargo.lock 5 additions, 0 deletionspolkadot/Cargo.lock
- polkadot/node/core/approval-voting/src/import.rs 1 addition, 0 deletionspolkadot/node/core/approval-voting/src/import.rs
- polkadot/node/core/approval-voting/src/lib.rs 1 addition, 0 deletionspolkadot/node/core/approval-voting/src/lib.rs
- polkadot/node/network/approval-distribution/Cargo.toml 3 additions, 0 deletionspolkadot/node/network/approval-distribution/Cargo.toml
- polkadot/node/network/approval-distribution/src/lib.rs 818 additions, 322 deletionspolkadot/node/network/approval-distribution/src/lib.rs
- polkadot/node/network/approval-distribution/src/metrics.rs 28 additions, 0 deletionspolkadot/node/network/approval-distribution/src/metrics.rs
- polkadot/node/network/approval-distribution/src/tests.rs 1176 additions, 1 deletionpolkadot/node/network/approval-distribution/src/tests.rs
- polkadot/node/network/bitfield-distribution/src/lib.rs 8 additions, 1 deletionpolkadot/node/network/bitfield-distribution/src/lib.rs
- polkadot/node/network/bridge/src/lib.rs 49 additions, 17 deletionspolkadot/node/network/bridge/src/lib.rs
- polkadot/node/network/collator-protocol/src/collator_side/mod.rs 1 addition, 1 deletion...t/node/network/collator-protocol/src/collator_side/mod.rs
- polkadot/node/network/collator-protocol/src/validator_side/mod.rs 1 addition, 1 deletion.../node/network/collator-protocol/src/validator_side/mod.rs
- polkadot/node/network/gossip-support/Cargo.toml 1 addition, 0 deletionspolkadot/node/network/gossip-support/Cargo.toml
- polkadot/node/network/gossip-support/src/lib.rs 126 additions, 53 deletionspolkadot/node/network/gossip-support/src/lib.rs
- polkadot/node/network/gossip-support/src/tests.rs 211 additions, 71 deletionspolkadot/node/network/gossip-support/src/tests.rs
- polkadot/node/network/statement-distribution/Cargo.toml 1 addition, 0 deletionspolkadot/node/network/statement-distribution/Cargo.toml
- polkadot/node/network/statement-distribution/src/lib.rs 8 additions, 1 deletionpolkadot/node/network/statement-distribution/src/lib.rs
- polkadot/node/network/statement-distribution/src/tests.rs 27 additions, 4 deletionspolkadot/node/network/statement-distribution/src/tests.rs
- polkadot/node/primitives/src/approval.rs 4 additions, 2 deletionspolkadot/node/primitives/src/approval.rs
- polkadot/node/subsystem-types/src/messages.rs 15 additions, 4 deletionspolkadot/node/subsystem-types/src/messages.rs
- polkadot/node/subsystem-types/src/messages/network_bridge_event.rs 30 additions, 6 deletions...node/subsystem-types/src/messages/network_bridge_event.rs
Please register or sign in to comment