📅 Original date posted:2022-10-05
📝 Original message:>> [Regarding bandwidth waste: I've pointed out in years past that
>> breaking the Bitcoin versioning scheme creates a requirement that any
>> unknown message type be considered valid. Up until a recently-deployed
>> protocol change, it had always been possible to validate messages by
>> type. I noticed recently that validating nodes have been dropping peers
>> at an increasing rate (a consequence of that deployment). Despite being
>> an undocumented compatibility break, it is now unfortunately a matter
>> of protocol that a peer must allow its peers to waste its bandwidth to
>> remain compatible - something which we should eliminate.]
>
> The only message listed as not being preceded by a bumped version number
> in:
>
> https://github.com/libbitcoin/libbitcoin-network/wiki/Protocol-Versioning
Good find, still a work in progress.
> is addrv2 (though addrv2 is gated on mutual exchange of sendaddrv2, so
> it's presumably the sendaddrv2 message at issue),
addrv2 is listed as the BIP title, the message that would cause the break is sendaddrv2 (quoted text).
> however since [0]
> sendaddrv2 messages are only sent to nodes advertising version 70016 or
> later (same as wtxidrelay).
I don’t see this constraint in BIP155. Do you mean that addrv2 support was released in Core at the same time as wtxidrelay, or that it is an undocumented version constraint implemented in Core?
> ADDRV2 was introduced May 20 2020 after the
> 0.20 branch, and SENDADDRV2 gating was merged Dec 9 2020 and included
> from 0.21.0rc3 onwards.
To clarify, there was no Core release of addrv2 without sendaddrv2 apart from 0.21 release candidates?
> [0] https://github.com/bitcoin/bitcoin/pull/20564
>
> I'm only seeing "bytesrecv_per_msg.*other*" entries for nodes advertising
> a version of 0.17 and 0.18,
> which I presume is due to REJECT messages (for taproot txs, perhaps?).
Ideally you should not be seeing reject messages as protocol “other”, as these are valid messages as of protocol version 70002, and they are excluded by negotiated version before that. While there is no requirement to send them (BIP61 only defines a new message type), they remain defined messages until removed by a future protocol version.
> Otherwise, I don't think there are any unexpected
> messages you should be receiving when advertising version 70015 or lower.
Yet nodes with an advertised protocol version of 70013 are receiving sendaddrv2. I've removed the IP address from the log extract below.
17:53:45.022347 DEBUG [network] Peer [x.x.x.x:8333] protocol version (70016) user agent: /Satoshi:0.21.0()/
17:53:45.022377 DEBUG [network] Negotiated protocol version (70013) for [x.x.x.x.135:8333]
17:53:45.022767 INFO [network] Connected outbound channel [x.x.x.x.135:8333]
17:53:45.022913 DEBUG [node] Ask [x.x.x.x:8333] for headers after [00000000000000000002e8c1c59fc86f721ba3a3294d2b1165597ddb910058e6]
17:53:45.023184 WARNING [network] Invalid sendaddrv2 payload from [x.x.x.x:8333] object does not exist
17:53:45.023317 DEBUG [network] Stop protocol version on [x.x.x.x:8333] object does not exist
17:53:45.023359 DEBUG [network] Outbound channel stopped [x.x.x.x:8333] success
To my knowledge the only other time we've seen consistent invalid message traffic on the network was during the work on BIP150 (withdrawn), at which point BIP150 nodes were being deployed on mainnet. I made comments here on the issue at the time, which as I recall were generally rejected in favor of forcing nodes to allow all invalid traffic. In any case BIP150 was withdrawn and BIP324 proposed, which fixes this particular issue (using a service bit).
Some argued at the time that allowance for invalid messages was a longstanding requirement in the protocol. I knew that this was not the case (except for BIP37, break documented in BIP60) because libbitcoin validates all messages, which led me to eventually document it. Recently I updated and posted that documentation (the github wiki link you found). This was a consequence of reviewing the Generic Package Relay proposal, which is also incompatible. In doing so I noticed this issue with BIP155 and BIP330 as well. This led us to check the logs for peer disconnects as a result of invalid messages, at which point the above was found to be an increasingly common occurrence.
Best,
e
> Cheers,
> aj
Eric Voskuil [ARCHIVE] /
npub1sg…ppx3c
2023-06-07 23:14:10
in reply to nevent1q…r87t
Author Public Key
npub1sgs97fe0n9wehe6zw7drcxdz4cy9yt9pfqjv8gasz5jlk4zezc0quppx3cPublished at
2023-06-07 23:14:10Event JSON
{
"id": "7bde7533c1b0d5fe7bd2893ce978f76ee0b2befa06697818e5d3f8c3cca87094",
"pubkey": "82205f272f995d9be742779a3c19a2ae08522ca14824c3a3b01525fb5459161e",
"created_at": 1686179650,
"kind": 1,
"tags": [
[
"e",
"05b7874cad20f8c3dcf4fbf4af657ea8217b28f4cd59bd5abdcd53c9d39053a7",
"",
"root"
],
[
"e",
"bd01b9a8064cf49039660791545f72fcad99e75db684411b161adc02d32eecf1",
"",
"reply"
],
[
"p",
"f0feda6ad58ea9f486e469f87b3b9996494363a26982b864667c5d8acb0542ab"
]
],
"content": "📅 Original date posted:2022-10-05\n📝 Original message:\u003e\u003e [Regarding bandwidth waste: I've pointed out in years past that\n\u003e\u003e breaking the Bitcoin versioning scheme creates a requirement that any\n\u003e\u003e unknown message type be considered valid. Up until a recently-deployed\n\u003e\u003e protocol change, it had always been possible to validate messages by\n\u003e\u003e type. I noticed recently that validating nodes have been dropping peers\n\u003e\u003e at an increasing rate (a consequence of that deployment). Despite being\n\u003e\u003e an undocumented compatibility break, it is now unfortunately a matter\n\u003e\u003e of protocol that a peer must allow its peers to waste its bandwidth to\n\u003e\u003e remain compatible - something which we should eliminate.]\n\u003e \n\u003e The only message listed as not being preceded by a bumped version number\n\u003e in:\n\u003e \n\u003e https://github.com/libbitcoin/libbitcoin-network/wiki/Protocol-Versioning\n\nGood find, still a work in progress.\n\n\u003e is addrv2 (though addrv2 is gated on mutual exchange of sendaddrv2, so\n\u003e it's presumably the sendaddrv2 message at issue),\n\naddrv2 is listed as the BIP title, the message that would cause the break is sendaddrv2 (quoted text).\n\n\u003e however since [0]\n\u003e sendaddrv2 messages are only sent to nodes advertising version 70016 or\n\u003e later (same as wtxidrelay).\n\nI don’t see this constraint in BIP155. Do you mean that addrv2 support was released in Core at the same time as wtxidrelay, or that it is an undocumented version constraint implemented in Core?\n\n\u003e ADDRV2 was introduced May 20 2020 after the\n\u003e 0.20 branch, and SENDADDRV2 gating was merged Dec 9 2020 and included\n\u003e from 0.21.0rc3 onwards.\n\nTo clarify, there was no Core release of addrv2 without sendaddrv2 apart from 0.21 release candidates?\n\n\u003e [0] https://github.com/bitcoin/bitcoin/pull/20564\n\u003e \n\u003e I'm only seeing \"bytesrecv_per_msg.*other*\" entries for nodes advertising\n\u003e a version of 0.17 and 0.18,\n\n\u003e which I presume is due to REJECT messages (for taproot txs, perhaps?).\n\nIdeally you should not be seeing reject messages as protocol “other”, as these are valid messages as of protocol version 70002, and they are excluded by negotiated version before that. While there is no requirement to send them (BIP61 only defines a new message type), they remain defined messages until removed by a future protocol version.\n\n\u003e Otherwise, I don't think there are any unexpected\n\u003e messages you should be receiving when advertising version 70015 or lower.\n\nYet nodes with an advertised protocol version of 70013 are receiving sendaddrv2. I've removed the IP address from the log extract below.\n\n17:53:45.022347 DEBUG [network] Peer [x.x.x.x:8333] protocol version (70016) user agent: /Satoshi:0.21.0()/\n17:53:45.022377 DEBUG [network] Negotiated protocol version (70013) for [x.x.x.x.135:8333]\n17:53:45.022767 INFO [network] Connected outbound channel [x.x.x.x.135:8333]\n17:53:45.022913 DEBUG [node] Ask [x.x.x.x:8333] for headers after [00000000000000000002e8c1c59fc86f721ba3a3294d2b1165597ddb910058e6]\n17:53:45.023184 WARNING [network] Invalid sendaddrv2 payload from [x.x.x.x:8333] object does not exist\n17:53:45.023317 DEBUG [network] Stop protocol version on [x.x.x.x:8333] object does not exist\n17:53:45.023359 DEBUG [network] Outbound channel stopped [x.x.x.x:8333] success\n\nTo my knowledge the only other time we've seen consistent invalid message traffic on the network was during the work on BIP150 (withdrawn), at which point BIP150 nodes were being deployed on mainnet. I made comments here on the issue at the time, which as I recall were generally rejected in favor of forcing nodes to allow all invalid traffic. In any case BIP150 was withdrawn and BIP324 proposed, which fixes this particular issue (using a service bit).\n\nSome argued at the time that allowance for invalid messages was a longstanding requirement in the protocol. I knew that this was not the case (except for BIP37, break documented in BIP60) because libbitcoin validates all messages, which led me to eventually document it. Recently I updated and posted that documentation (the github wiki link you found). This was a consequence of reviewing the Generic Package Relay proposal, which is also incompatible. In doing so I noticed this issue with BIP155 and BIP330 as well. This led us to check the logs for peer disconnects as a result of invalid messages, at which point the above was found to be an increasingly common occurrence.\n\nBest,\ne\n\n\u003e Cheers,\n\u003e aj",
"sig": "f4615a1f2ff8df89c97660ce3d9f68fb8dce1ecb271532c31b9345258825ee4285875c8fe2a4e2eb5f81390b963969c1c06721b163b9be5a148c4bdab84d17e3"
}