📅 Original date posted:2015-08-06
📝 Original message:No, don't think so, the protocol is, essentially, relay transactions, when you get a block, send header, iterate over transactions, for each, either use two bytes for nth-recent-transaction-relayed, use 0xffff-3-byte-length-transaction-data. There are quite a few implementation details, and lots of things could be improved, but that is pretty much how it works.
Matt
On August 6, 2015 7:16:56 PM GMT+02:00, Sergio Demian Lerner via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>Is there any up to date documentation about TheBlueMatt relay network
>including what kind of block compression it is currently doing? (apart
>from
>the source code)
>
>Regards, Sergio.
>
>On Wed, Aug 5, 2015 at 7:14 PM, Gregory Maxwell via bitcoin-dev <
>bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>> On Wed, Aug 5, 2015 at 9:19 PM, Arnoud Kouwenhoven - Pukaki Corp
>> <arnoud at pukaki.bz> wrote:
>> > Thanks for this (direct) feedback. It would make sense that if
>blocks
>> can be
>> > submitted using ~5kb packets, that no further optimizations would
>be
>> needed
>> > at this point. I will look into the relay network transmission
>protocol
>> to
>> > understand how it works!
>> >
>> > I hear that you are saying that this network solves speed of
>transmission
>> > and thereby (technical) block size issues. Presumably it would
>solve
>> speed
>> > of block validation too by prevalidating transactions.
>>
>>
>> Correct. Bitcoin Core has cached validation for many years now... if
>> not for that and other optimizations, things would be really broken
>> right now. :)
>>
>> > Assuming this is all
>> > true, and I have no reason to doubt that at this point, I do not
>> understand
>> > why there is any discussion at all about the (technical) impact of
>large
>> > blocks, why there are large numbers of miners building on invalid
>blocks
>> > (SPV mining, https://bitcoin.org/en/alert/2015-07-04-spv-mining),
>or why
>> > there is any discussion about the speed of block validation (cpu
>> processing
>> > time to verify blocks and transactions in blocks being a
>limitation).
>>
>> I'm also mystified by a lot of the large block discussion, much of it
>> is completely divorced from the technology as deployed; much less
>what
>> we-- in industry-- know to be possible. I don't blame you or anyone
>in
>> particular on this; it's a new area and we don't yet know what we
>need
>> to know to know what we need to know; or to the extent that we do it
>> hasn't had time to get effectively communicated.
>>
>> The technical/security implications of larger blocks are related to
>> other things than propagation time, if you assume people are using
>the
>> available efficient relay protocol (or better).
>>
>> SPV mining is a bit of a misnomer (If I coined the term, I'm sorry).
>> What these parties are actually doing is blinding mining on top of
>> other pools' stratum work. You can think of it as sub-pooling with
>> hopping onto whatever pool has the highest block (I'll call it VFSSP
>> in this post-- validation free stratum subpooling). It's very easy
>to
>> implement, and there are other considerations.
>>
>> It was initially deployed at a time when a single pool in Europe has
>> amassed more than half of the hashrate. This pool had propagation
>> problems and a very high orphan rate, it may have (perhaps
>> unintentionally) been performing a selfish mining attack; mining off
>> their stratum work was an easy fix which massively cut down the
>orphan
>> rates for anyone who did it. This was before the relay network
>> protocol existed (the fact that all the hashpower was consolidating
>on
>> a single pool was a major motivation for creating it).
>>
>> VFSSP also cuts through a number of practical issues miners have had:
>> Miners that run their own bitcoin nodes in far away colocation
>> (>100ms) due to local bandwidth or connectivity issues (censored
>> internet); relay network hubs not being anywhere near by due to
>> strange internet routing (e.g. japan to china going via the US for
>...
>> reasons...); the CreateNewBlock() function being very slow and
>> unoptimized, etc. There are many other things like this-- and VFSSP
>> avoids them causing delays even when you don't understand them or
>know
>> about them. So even when they're easily fixed the VFSSP is a more
>> general workaround.
>>
>> Mining operations are also usually operated in a largely fire and
>> forget manner. There is a long history in (esp pooled) mining where
>> someone sets up an operation and then hardly maintains it after the
>> fact... so some of the use of VFSSP appears to just be inertia-- we
>> have better solutions now, but they they work to deploy and changing
>> things involves risk (which is heightened by a lack of good
>> monitoring-- participants learn they are too latent by observing
>> orphaned blocks at a cost of 25 BTC each).
>>
>> One of the frustrating things about incentives in this space is that
>> bad outcomes are possible even when they're not necessary. E.g. if a
>> miner can lower their orphan rate by deploying a new protocol (or
>> simply fixing some faulty hardware in their infrastructure, like
>> Bitcoin nodes running on cheap VPSes with remote storage) OR they
>can
>> lower their orphan rate by pointing their hashpower at a free
>> centeralized pool, they're likely to do the latter because it takes
>> less effort.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150806/34f9094e/attachment-0001.html>;
Matt Corallo [ARCHIVE] /
npub1e4…jxmcu
2023-06-07 15:45:29
in reply to nevent1q…p2qk
Author Public Key
npub1e46n428mcyfwznl7nlsf6d3s7rhlwm9x3cmkuqzt3emmdpadmkaqqjxmcuPublished at
2023-06-07 15:45:29Event JSON
{
"id": "7a7ba365739f5abf770862b73ede0b97ddaec4de53138de063f69480aa4143d7",
"pubkey": "cd753aa8fbc112e14ffe9fe09d3630f0eff76ca68e376e004b8e77b687adddba",
"created_at": 1686152729,
"kind": 1,
"tags": [
[
"e",
"158880a596c782b1fb4c5a60a229c1c08a5de1f9a4400f68dd2a5506a1a8001b",
"",
"root"
],
[
"e",
"65d846c344b2bcca327c7bd32c0d9e008903baaef2f07a570e03e5fd76dc2938",
"",
"reply"
],
[
"p",
"cd753aa8fbc112e14ffe9fe09d3630f0eff76ca68e376e004b8e77b687adddba"
]
],
"content": "📅 Original date posted:2015-08-06\n📝 Original message:No, don't think so, the protocol is, essentially, relay transactions, when you get a block, send header, iterate over transactions, for each, either use two bytes for nth-recent-transaction-relayed, use 0xffff-3-byte-length-transaction-data. There are quite a few implementation details, and lots of things could be improved, but that is pretty much how it works.\n\nMatt\n\nOn August 6, 2015 7:16:56 PM GMT+02:00, Sergio Demian Lerner via bitcoin-dev \u003cbitcoin-dev at lists.linuxfoundation.org\u003e wrote:\n\u003eIs there any up to date documentation about TheBlueMatt relay network\n\u003eincluding what kind of block compression it is currently doing? (apart\n\u003efrom\n\u003ethe source code)\n\u003e\n\u003eRegards, Sergio.\n\u003e\n\u003eOn Wed, Aug 5, 2015 at 7:14 PM, Gregory Maxwell via bitcoin-dev \u003c\n\u003ebitcoin-dev at lists.linuxfoundation.org\u003e wrote:\n\u003e\n\u003e\u003e On Wed, Aug 5, 2015 at 9:19 PM, Arnoud Kouwenhoven - Pukaki Corp\n\u003e\u003e \u003carnoud at pukaki.bz\u003e wrote:\n\u003e\u003e \u003e Thanks for this (direct) feedback. It would make sense that if\n\u003eblocks\n\u003e\u003e can be\n\u003e\u003e \u003e submitted using ~5kb packets, that no further optimizations would\n\u003ebe\n\u003e\u003e needed\n\u003e\u003e \u003e at this point. I will look into the relay network transmission\n\u003eprotocol\n\u003e\u003e to\n\u003e\u003e \u003e understand how it works!\n\u003e\u003e \u003e\n\u003e\u003e \u003e I hear that you are saying that this network solves speed of\n\u003etransmission\n\u003e\u003e \u003e and thereby (technical) block size issues. Presumably it would\n\u003esolve\n\u003e\u003e speed\n\u003e\u003e \u003e of block validation too by prevalidating transactions.\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e Correct. Bitcoin Core has cached validation for many years now... if\n\u003e\u003e not for that and other optimizations, things would be really broken\n\u003e\u003e right now. :)\n\u003e\u003e\n\u003e\u003e \u003e Assuming this is all\n\u003e\u003e \u003e true, and I have no reason to doubt that at this point, I do not\n\u003e\u003e understand\n\u003e\u003e \u003e why there is any discussion at all about the (technical) impact of\n\u003elarge\n\u003e\u003e \u003e blocks, why there are large numbers of miners building on invalid\n\u003eblocks\n\u003e\u003e \u003e (SPV mining, https://bitcoin.org/en/alert/2015-07-04-spv-mining),\n\u003eor why\n\u003e\u003e \u003e there is any discussion about the speed of block validation (cpu\n\u003e\u003e processing\n\u003e\u003e \u003e time to verify blocks and transactions in blocks being a\n\u003elimitation).\n\u003e\u003e\n\u003e\u003e I'm also mystified by a lot of the large block discussion, much of it\n\u003e\u003e is completely divorced from the technology as deployed; much less\n\u003ewhat\n\u003e\u003e we-- in industry-- know to be possible. I don't blame you or anyone\n\u003ein\n\u003e\u003e particular on this; it's a new area and we don't yet know what we\n\u003eneed\n\u003e\u003e to know to know what we need to know; or to the extent that we do it\n\u003e\u003e hasn't had time to get effectively communicated.\n\u003e\u003e\n\u003e\u003e The technical/security implications of larger blocks are related to\n\u003e\u003e other things than propagation time, if you assume people are using\n\u003ethe\n\u003e\u003e available efficient relay protocol (or better).\n\u003e\u003e\n\u003e\u003e SPV mining is a bit of a misnomer (If I coined the term, I'm sorry).\n\u003e\u003e What these parties are actually doing is blinding mining on top of\n\u003e\u003e other pools' stratum work. You can think of it as sub-pooling with\n\u003e\u003e hopping onto whatever pool has the highest block (I'll call it VFSSP\n\u003e\u003e in this post-- validation free stratum subpooling). It's very easy\n\u003eto\n\u003e\u003e implement, and there are other considerations.\n\u003e\u003e\n\u003e\u003e It was initially deployed at a time when a single pool in Europe has\n\u003e\u003e amassed more than half of the hashrate. This pool had propagation\n\u003e\u003e problems and a very high orphan rate, it may have (perhaps\n\u003e\u003e unintentionally) been performing a selfish mining attack; mining off\n\u003e\u003e their stratum work was an easy fix which massively cut down the\n\u003eorphan\n\u003e\u003e rates for anyone who did it. This was before the relay network\n\u003e\u003e protocol existed (the fact that all the hashpower was consolidating\n\u003eon\n\u003e\u003e a single pool was a major motivation for creating it).\n\u003e\u003e\n\u003e\u003e VFSSP also cuts through a number of practical issues miners have had:\n\u003e\u003e Miners that run their own bitcoin nodes in far away colocation\n\u003e\u003e (\u003e100ms) due to local bandwidth or connectivity issues (censored\n\u003e\u003e internet); relay network hubs not being anywhere near by due to\n\u003e\u003e strange internet routing (e.g. japan to china going via the US for\n\u003e...\n\u003e\u003e reasons...); the CreateNewBlock() function being very slow and\n\u003e\u003e unoptimized, etc. There are many other things like this-- and VFSSP\n\u003e\u003e avoids them causing delays even when you don't understand them or\n\u003eknow\n\u003e\u003e about them. So even when they're easily fixed the VFSSP is a more\n\u003e\u003e general workaround.\n\u003e\u003e\n\u003e\u003e Mining operations are also usually operated in a largely fire and\n\u003e\u003e forget manner. There is a long history in (esp pooled) mining where\n\u003e\u003e someone sets up an operation and then hardly maintains it after the\n\u003e\u003e fact... so some of the use of VFSSP appears to just be inertia-- we\n\u003e\u003e have better solutions now, but they they work to deploy and changing\n\u003e\u003e things involves risk (which is heightened by a lack of good\n\u003e\u003e monitoring-- participants learn they are too latent by observing\n\u003e\u003e orphaned blocks at a cost of 25 BTC each).\n\u003e\u003e\n\u003e\u003e One of the frustrating things about incentives in this space is that\n\u003e\u003e bad outcomes are possible even when they're not necessary. E.g. if a\n\u003e\u003e miner can lower their orphan rate by deploying a new protocol (or\n\u003e\u003e simply fixing some faulty hardware in their infrastructure, like\n\u003e\u003e Bitcoin nodes running on cheap VPSes with remote storage) OR they\n\u003ecan\n\u003e\u003e lower their orphan rate by pointing their hashpower at a free\n\u003e\u003e centeralized pool, they're likely to do the latter because it takes\n\u003e\u003e less effort.\n\u003e\u003e _______________________________________________\n\u003e\u003e bitcoin-dev mailing list\n\u003e\u003e bitcoin-dev at lists.linuxfoundation.org\n\u003e\u003e https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev\n\u003e\u003e\n\u003e\n\u003e\n\u003e------------------------------------------------------------------------\n\u003e\n\u003e_______________________________________________\n\u003ebitcoin-dev mailing list\n\u003ebitcoin-dev at lists.linuxfoundation.org\n\u003ehttps://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150806/34f9094e/attachment-0001.html\u003e",
"sig": "f699054a1cc5e5c65f71cbcc914e12ad8f922b51d017000f6b8af46974b77c9b6c099bc1959ca6dc2ae879c29e4f4e9604cc2b266a2186dcd2de0df2693d03d1"
}