📅 Original date posted:2015-09-18
📝 Original message:Currently, when a new node wants to join the network, it needs to retrieve the entire blockchain history, starting from January 2009 and up until now, in order to derive a UTXO set that it can verify new blocks/transactions against. With a blockchain size of 40GB and a UTXO size of around 1GB, the extra bandwidth required is significant, and will keep increasing indefinitely. If a newly mined block were to include the UTXO set hash of the chain up until the previous block — the hash of the UTXO set on top of which this block builds — then new nodes, who want to know whether a transaction is valid, would be able to acquire the UTXO set in a trustless manner, by only verifying proof-of-work headers, and knowing that a block with an invalid UTXO set hash would be rejected.
I’m not talking about calculating a complicated tree structure from the UTXO set, which would put further burden on already burdened Bitcoin Core nodes. We simply include the hash of the current UTXO set in a newly created block, such that the transactions in the new block build *on top* of the UTXO set whose hash is specified. This actually alleviates Bitcoin Core nodes, as it will now become possible for nodes without the entire blockchain to answer SPV queries (by retrieving the UTXO set trustlessly and using this to answer queries). It also saves bandwidth for Bitcore Core nodes, who only need to send roughly 1GB of data, in order to synchronise a node, rather than 40GB+. I will continue to run a full Bitcoin Core node, saving the entire blockchain history, but it shouldn’t be a requirement to hold the entire transaction history in order to start verifying new transactions.
As far as I can see, this also forces miners to actually maintain an UTXO set, rather than just build on top of the chain with the most proof-of-work. Producing a UTXO set and verifying a block against a chain is the same thing, so by including the hash of the UTXO set we force miners to verify the block that they want to build on top of.
Am I missing something obvious, because as far as I can see, this solves the problem of quadratic time complexity for initial sync: http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s
The only added step to verifying a block is to hash the UTXO set. So it does require additional computation, but most modern CPUs have a SHA256 throughput of around 500 MB/s, which means it takes only two seconds to hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of initial syncing, in my opinion.
/Rune
Rune Kjær Svendsen [ARCHIVE] /
npub1ss…k53fg
2023-06-07 17:40:55
in reply to nevent1q…cp5q
Author Public Key
npub1ssutaataqhl7xxp4lw6jutuhfuq5x48cf0xsxpgsxdkxkf9rl9wqnk53fgPublished at
2023-06-07 17:40:55Event JSON
{
"id": "e986d9fc783a16e47a45e10b0288a5247983e726b38013185dd7dda2546e7286",
"pubkey": "8438bef57d05ffe31835fbb52e2f974f014354f84bcd030510336c6b24a3f95c",
"created_at": 1686159655,
"kind": 1,
"tags": [
[
"e",
"7fadb5378c1c44e78cc5de59daa2e951bce7b919506a3d6cff2bcf05fa79c75d",
"",
"reply"
],
[
"p",
"a23dbf6c6cc83e14cc3df4e56cc71845f611908084cfe620e83e40c06ccdd3d0"
]
],
"content": "📅 Original date posted:2015-09-18\n📝 Original message:Currently, when a new node wants to join the network, it needs to retrieve the entire blockchain history, starting from January 2009 and up until now, in order to derive a UTXO set that it can verify new blocks/transactions against. With a blockchain size of 40GB and a UTXO size of around 1GB, the extra bandwidth required is significant, and will keep increasing indefinitely. If a newly mined block were to include the UTXO set hash of the chain up until the previous block — the hash of the UTXO set on top of which this block builds — then new nodes, who want to know whether a transaction is valid, would be able to acquire the UTXO set in a trustless manner, by only verifying proof-of-work headers, and knowing that a block with an invalid UTXO set hash would be rejected.\n\nI’m not talking about calculating a complicated tree structure from the UTXO set, which would put further burden on already burdened Bitcoin Core nodes. We simply include the hash of the current UTXO set in a newly created block, such that the transactions in the new block build *on top* of the UTXO set whose hash is specified. This actually alleviates Bitcoin Core nodes, as it will now become possible for nodes without the entire blockchain to answer SPV queries (by retrieving the UTXO set trustlessly and using this to answer queries). It also saves bandwidth for Bitcore Core nodes, who only need to send roughly 1GB of data, in order to synchronise a node, rather than 40GB+. I will continue to run a full Bitcoin Core node, saving the entire blockchain history, but it shouldn’t be a requirement to hold the entire transaction history in order to start verifying new transactions.\n\nAs far as I can see, this also forces miners to actually maintain an UTXO set, rather than just build on top of the chain with the most proof-of-work. Producing a UTXO set and verifying a block against a chain is the same thing, so by including the hash of the UTXO set we force miners to verify the block that they want to build on top of.\n\nAm I missing something obvious, because as far as I can see, this solves the problem of quadratic time complexity for initial sync: http://www.youtube.com/watch?v=TgjrS-BPWDQ\u0026t=2h02m12s\n\nThe only added step to verifying a block is to hash the UTXO set. So it does require additional computation, but most modern CPUs have a SHA256 throughput of around 500 MB/s, which means it takes only two seconds to hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of initial syncing, in my opinion.\n\n/Rune",
"sig": "a36ade29c4fae0fe6e5a3a47387e7bd6d17aa91b023f0de304f79b650424e1a0e713716151b685efac13068b4a801a2f81e356c3d47db75076f9917760ca06e1"
}