← 输入框|github.com|刷新
Skip to content

Add multi-peer parallel block download during IBD #27

Open

Description

@pzafonte

Summary

IBD currently uses a single peer, bottlenecking sync on network latency. This adds parallel block download from multiple peers while validating sequentially through process_block.

Proof-of-concept: https://github.com/pzafonte/kernel-node/tree/multi-peer-ibd

Design

  • PeerManager spawns N threads (configurable --maxpeers, default 4) with auto-reconnect
  • Shared download queue (Mutex<VecDeque<BlockHash>>) populated after header sync
  • In-flight set (Mutex<HashSet<BlockHash>>) prevents duplicate requests across peers
  • Per-peer state machine: AwaitingHeaders -> AwaitingInv / AwaitingBlock with 16-block batches
  • Single validation thread via sync_channel(32) chainman is not thread-safe for concurrent process_block
  • First peer syncs headers; late peers skip straight to block download (AtomicBool flag)
  • Stalled peers (30s timeout) are disconnected; unreceived blocks re-enqueued for other peers
  • Logarithmic block locators for getheaders/getblocks

This is a proof-of-concept implementation. Design tradeoffs favor simplicity over production-readiness. Each is documented with an inline // Tradeoff: comment explaining the cost and what a production implementation would do differently. There are probably other enhancements that could be made too.

Test Coverage

27 tests: 21 unit tests (batch popping, message construction, deduplication, state machine) + 6 integration tests with regtest ChainstateManager.

Activity

  1. sedited commented on Feb 26, 2026

    @sedited
    Collaborator

    Very nice! Feel free to just PR it. That makes it easier for other people to submit review too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions