IBD headers optimizations

[freshair18]

During IBD, headers are initially supplied to the node at the headers only stage, in order to determine the most advanced selected tip quickly.
The blocks download stage later asks for the full blocks. However currently the headers too are redownloaded despite already being available.

The reasoning for this is mostly due to simplicity and legacy logic from Bitcoin, where headers are incredibly small. In Kaspa the headers consist of multiple parents' hashes instead of 1, and moreso contain higher level parents' hashes as well. Headers are still small compared to body, but no longer negligibly small, resulting in a small but unnecessary elongation of IBD time and extra bandwidth consumption.

Optimization proposal

Implement a new p2p message to provide block body only, with structure roughly like:
struct block_body{ hash: Hash, txs: Vec<Transaction>, },

and have nodes receiving this message construct the block body on their own and match and verify it with the already available header.

[coderofstuff]

Can you add some rough numbers on how much bandwidth is saved here in the context of 10bps?

This would require changes in:

• p2p
• body processor

[miningexperiments]

during ibd sync_headers processing the cpu and bandwith are in quite low. Some testing I did for one peer:

1. If I remove all processing the download speed increases from 0.2 to 1 Mbps for that specific peer; e.g. simply downloading chunks with next(). I assume this was the max upload speed for the peer.
2. Then I tried collecting these into 500 element chunks first as a buffer to see the cpu utilization. It increased, but, maybe from 20% to 30% with multicores.

• With some late night testing it seemed that each chunks is 0.2Mb, with about 1022-1025 calls to validate_insert_block each.
• Each call also creates a tokio onechannel, so after downloading 1200chunks it was about 1M channels.
• Each calls also ends up adding to thread_spawn pool,

The IBD seems to work the same way for block processing and there cpu is at 100% use and bandwith massively increased for downloads (multiple peers i suppose?).

I am wondering if there is too much overhead for headers processing to go through?

rusty-kaspa/protocol/flows/src/v5/ibd/flow.rs

Line 381 in ac677a0

async fn sync_headers(

rusty-kaspa/consensus/src/consensus/mod.rs

Line 328 in cdd4379

fn validate_and_insert_block_impl(

rusty-kaspa/consensus/src/pipeline/header_processor/processor.rs

Line 212 in cdd4379

pub fn worker(self: &Arc<HeaderProcessor>) {