📅 Original date posted:2013-06-20
📝 Original message:I personally don't treat the relay field as optional, i.e. it is there as
0x01 if it is set. Otherwise, it is simply a trailing zero byte. Hence, the
right way of reading the packet as with any network packet is to first
retrieve the header information, get the actual payload length, then parse
the payload accordingly. I can also choose to include 0x00 for my relay
field in my outgoing packet and reflect that accordingly in my length field
in the header.
On Thu, Jun 20, 2013 at 4:20 PM, Turkey Breast <turkeybreast at yahoo.com>wrote:
> I never said that Bitcoin message field lengths should always be the same.
> But before this change they certainly were constant per protocol version.
> All I'm saying is that optional lengths shouldn't be used (a field exists
> or not) and for every field change, the protocol version should be upgraded.
>
> Now that fRelayTxes is part of the protocol, the version number should be
> upgraded to reflect this fact.
>
> ------------------------------
> *From:* Mike Hearn <mike at plan99.net>
> *To:* Paul Lyon <pmlyon at hotmail.ca>
> *Cc:* Turkey Breast <turkeybreast at yahoo.com>; "
> bitcoin-development at lists.sourceforge.net" <
> bitcoin-development at lists.sourceforge.net>
> *Sent:* Wednesday, June 19, 2013 3:20 PM
> *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version message
>
> If you want to criticise the Bitcoin protocol for sloppyness, the variable
> length of some messages isn't where I'd start.
>
> Note that ping has the same issue, its length has changed over time to
> include the nonce.
>
> If your parser can't handle that kind of thing, you need to fix it. The
> protocol has always worked that way.
>
>
>
> On Wed, Jun 19, 2013 at 3:03 PM, Paul Lyon <pmlyon at hotmail.ca> wrote:
>
> I’m also running into this exact same issue with my parser, now I
> understand why the relay field behavior I was seeing doesn’t match the wiki.
>
> So to parse a version message, you can’t rely on the protocol version? You
> have to know how long the payload is, and then parse the message
> accordingly? I agree with Turkey Breast, this seems a bit sloppy to me.
>
> Paul
>
> P.S. I’ve never used a dev mailing list before and I want to get involved
> with the Bitcoin dev community, so let me know if I’m horribly violating
> any mailing list etiquette. 😊
>
> *From:* Mike Hearn
> *Sent:* Wednesday, June 19, 2013 7:43 AM
> *To:* Turkey Breast
> *Cc:* bitcoin-development at lists.sourceforge.net
>
> Bitcoin-Qt on master does send it now although it doesn't affect anything,
> but as old pre-filtering versions will continue to exist, you'll always
> have to be able to deserialize version messages without it.
>
> Bitcoin version messages have always had variable length, look at how the
> code is written in main.cpp. If you didn't experience issues until now all
> it means is that no sufficiently old nodes were talking to yours.
>
> The standard does not say it should appear. Read it again - BIP 37 says
> about the new version message field:
> If false then broadcast transactions will not be announced until a
> filter{load,add,clear} command is received. *If missing or true*, no
> change in protocol behaviour occurs.
>
>
> On Wed, Jun 19, 2013 at 12:33 PM, Turkey Breast <turkeybreast at yahoo.com>wrote:
>
> It's a problem if you work with iterators to deserialize the byte stream.
> Even failing that, it's just sloppy programming. What happens in the future
> when new fields are added to the version message? It's not a big deal to
> say that this protocol version has X number of fields, that (higher)
> protocol version message has X + N number of fields. Deterministic number
> of fields per protocol version is sensical and how Bitcoin has been for a
> long time.
>
> And yes, it was a problem for me that caused a lot of confusion why this
> byte didn't exist in many version messages despite the standard saying it
> should and the code in bitcoind indicating it should. Nowhere was this
> written. It doesn't help other implementations to have an unclear behaviour
> that depends on some magic from one implementation.
>
> ------------------------------
> *From:* Mike Hearn <mike at plan99.net>
> *To:* Turkey Breast <turkeybreast at yahoo.com>
> *Cc:* "bitcoin-development at lists.sourceforge.net" <
> bitcoin-development at lists.sourceforge.net>
> *Sent:* Wednesday, June 19, 2013 11:39 AM
>
> *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version message
>
> It has to be optional because old clients don't send it, obviously.
>
> Why is this even an issue? There's no problem with variable length
> messages in any codebase that I'm aware of. Is this solving some actual
> problem?
>
>
> On Wed, Jun 19, 2013 at 12:30 AM, Turkey Breast <turkeybreast at yahoo.com>wrote:
>
> That's me. I never said to make all messages fixed length. I said to make
> a fixed number of fields per protocol. So given a protocol version number,
> you know the number of fields in a message. This is not only easier for
> parsing messages, but just good practice. I don't see why a 1 byte flag
> needs to be optional anyway.
>
> ------------------------------
> *From:* Mike Hearn <mike at plan99.net>
> *To:* Turkey Breast <turkeybreast at yahoo.com>
> *Cc:* Bitcoin Dev <bitcoin-development at lists.sourceforge.net>
> *Sent:* Tuesday, June 18, 2013 9:48 PM
> *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version message
>
> It's not a bug (although there was recently a change to make bitcoind/qt
> always send this field anyway).
>
> I don't know where Amir is going with BIP 60. Version messages have always
> been variable length. There's nothing inherent in the Bitcoin protocol that
> says all messages are fixed length, indeed, tx messages are allowed to have
> arbitrary data appended after them that gets relayed.
>
>
> On Tue, Jun 18, 2013 at 7:45 PM, Turkey Breast <turkeybreast at yahoo.com>wrote:
>
> See this BIP. I'm not sure if this is a bug or what, but it would be good
> if messages always had a fixed number of fields per protocol version.
>
> https://en.bitcoin.it/wiki/BIP_0060#Code_Updates
>
> This BIP details everything that needs to be done and proposes a protocol
> upgrade.
>
>
> ------------------------------------------------------------------------------
> This SF.net <http://sf.net/>; email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20130620/0683b160/attachment.html>;
Addy Yeow [ARCHIVE] /
npub19z…f0jzn
2023-06-07 15:03:44
in reply to nevent1q…l5cw
Author Public Key
npub19zr8ffwpjn38e034sumg72g306uquf94h0gsrqlz4p4vrg6m3w8sqf0jznPublished at
2023-06-07 15:03:44Event JSON
{
"id": "7bb13639a706f0a9bde7267f60e880ecd5279b0471ab5e22bdfd1882ca91a2ae",
"pubkey": "288674a5c194e27cbe3587368f29117eb80e24b5bbd10183e2a86ac1a35b8b8f",
"created_at": 1686150224,
"kind": 1,
"tags": [
[
"e",
"ceba0091ad9a2fc6abd2e8b950bead35c2f78a7f07c36f1b39214a94ffed3e49",
"",
"root"
],
[
"e",
"fe532ae3ceabb2cb564b92998c8933754041e56ec9826ce39afdb4dc32d897c3",
"",
"reply"
],
[
"p",
"f2c95df3766562e3b96b79a0254881c59e8639f23987846961cf55412a77f6f2"
]
],
"content": "📅 Original date posted:2013-06-20\n📝 Original message:I personally don't treat the relay field as optional, i.e. it is there as\n0x01 if it is set. Otherwise, it is simply a trailing zero byte. Hence, the\nright way of reading the packet as with any network packet is to first\nretrieve the header information, get the actual payload length, then parse\nthe payload accordingly. I can also choose to include 0x00 for my relay\nfield in my outgoing packet and reflect that accordingly in my length field\nin the header.\n\n\nOn Thu, Jun 20, 2013 at 4:20 PM, Turkey Breast \u003cturkeybreast at yahoo.com\u003ewrote:\n\n\u003e I never said that Bitcoin message field lengths should always be the same.\n\u003e But before this change they certainly were constant per protocol version.\n\u003e All I'm saying is that optional lengths shouldn't be used (a field exists\n\u003e or not) and for every field change, the protocol version should be upgraded.\n\u003e\n\u003e Now that fRelayTxes is part of the protocol, the version number should be\n\u003e upgraded to reflect this fact.\n\u003e\n\u003e ------------------------------\n\u003e *From:* Mike Hearn \u003cmike at plan99.net\u003e\n\u003e *To:* Paul Lyon \u003cpmlyon at hotmail.ca\u003e\n\u003e *Cc:* Turkey Breast \u003cturkeybreast at yahoo.com\u003e; \"\n\u003e bitcoin-development at lists.sourceforge.net\" \u003c\n\u003e bitcoin-development at lists.sourceforge.net\u003e\n\u003e *Sent:* Wednesday, June 19, 2013 3:20 PM\n\u003e *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version message\n\u003e\n\u003e If you want to criticise the Bitcoin protocol for sloppyness, the variable\n\u003e length of some messages isn't where I'd start.\n\u003e\n\u003e Note that ping has the same issue, its length has changed over time to\n\u003e include the nonce.\n\u003e\n\u003e If your parser can't handle that kind of thing, you need to fix it. The\n\u003e protocol has always worked that way.\n\u003e\n\u003e\n\u003e\n\u003e On Wed, Jun 19, 2013 at 3:03 PM, Paul Lyon \u003cpmlyon at hotmail.ca\u003e wrote:\n\u003e\n\u003e I’m also running into this exact same issue with my parser, now I\n\u003e understand why the relay field behavior I was seeing doesn’t match the wiki.\n\u003e\n\u003e So to parse a version message, you can’t rely on the protocol version? You\n\u003e have to know how long the payload is, and then parse the message\n\u003e accordingly? I agree with Turkey Breast, this seems a bit sloppy to me.\n\u003e\n\u003e Paul\n\u003e\n\u003e P.S. I’ve never used a dev mailing list before and I want to get involved\n\u003e with the Bitcoin dev community, so let me know if I’m horribly violating\n\u003e any mailing list etiquette. 😊\n\u003e\n\u003e *From:* Mike Hearn\n\u003e *Sent:* Wednesday, June 19, 2013 7:43 AM\n\u003e *To:* Turkey Breast\n\u003e *Cc:* bitcoin-development at lists.sourceforge.net\n\u003e\n\u003e Bitcoin-Qt on master does send it now although it doesn't affect anything,\n\u003e but as old pre-filtering versions will continue to exist, you'll always\n\u003e have to be able to deserialize version messages without it.\n\u003e\n\u003e Bitcoin version messages have always had variable length, look at how the\n\u003e code is written in main.cpp. If you didn't experience issues until now all\n\u003e it means is that no sufficiently old nodes were talking to yours.\n\u003e\n\u003e The standard does not say it should appear. Read it again - BIP 37 says\n\u003e about the new version message field:\n\u003e If false then broadcast transactions will not be announced until a\n\u003e filter{load,add,clear} command is received. *If missing or true*, no\n\u003e change in protocol behaviour occurs.\n\u003e\n\u003e\n\u003e On Wed, Jun 19, 2013 at 12:33 PM, Turkey Breast \u003cturkeybreast at yahoo.com\u003ewrote:\n\u003e\n\u003e It's a problem if you work with iterators to deserialize the byte stream.\n\u003e Even failing that, it's just sloppy programming. What happens in the future\n\u003e when new fields are added to the version message? It's not a big deal to\n\u003e say that this protocol version has X number of fields, that (higher)\n\u003e protocol version message has X + N number of fields. Deterministic number\n\u003e of fields per protocol version is sensical and how Bitcoin has been for a\n\u003e long time.\n\u003e\n\u003e And yes, it was a problem for me that caused a lot of confusion why this\n\u003e byte didn't exist in many version messages despite the standard saying it\n\u003e should and the code in bitcoind indicating it should. Nowhere was this\n\u003e written. It doesn't help other implementations to have an unclear behaviour\n\u003e that depends on some magic from one implementation.\n\u003e\n\u003e ------------------------------\n\u003e *From:* Mike Hearn \u003cmike at plan99.net\u003e\n\u003e *To:* Turkey Breast \u003cturkeybreast at yahoo.com\u003e\n\u003e *Cc:* \"bitcoin-development at lists.sourceforge.net\" \u003c\n\u003e bitcoin-development at lists.sourceforge.net\u003e\n\u003e *Sent:* Wednesday, June 19, 2013 11:39 AM\n\u003e\n\u003e *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version message\n\u003e\n\u003e It has to be optional because old clients don't send it, obviously.\n\u003e\n\u003e Why is this even an issue? There's no problem with variable length\n\u003e messages in any codebase that I'm aware of. Is this solving some actual\n\u003e problem?\n\u003e\n\u003e\n\u003e On Wed, Jun 19, 2013 at 12:30 AM, Turkey Breast \u003cturkeybreast at yahoo.com\u003ewrote:\n\u003e\n\u003e That's me. I never said to make all messages fixed length. I said to make\n\u003e a fixed number of fields per protocol. So given a protocol version number,\n\u003e you know the number of fields in a message. This is not only easier for\n\u003e parsing messages, but just good practice. I don't see why a 1 byte flag\n\u003e needs to be optional anyway.\n\u003e\n\u003e ------------------------------\n\u003e *From:* Mike Hearn \u003cmike at plan99.net\u003e\n\u003e *To:* Turkey Breast \u003cturkeybreast at yahoo.com\u003e\n\u003e *Cc:* Bitcoin Dev \u003cbitcoin-development at lists.sourceforge.net\u003e\n\u003e *Sent:* Tuesday, June 18, 2013 9:48 PM\n\u003e *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version message\n\u003e\n\u003e It's not a bug (although there was recently a change to make bitcoind/qt\n\u003e always send this field anyway).\n\u003e\n\u003e I don't know where Amir is going with BIP 60. Version messages have always\n\u003e been variable length. There's nothing inherent in the Bitcoin protocol that\n\u003e says all messages are fixed length, indeed, tx messages are allowed to have\n\u003e arbitrary data appended after them that gets relayed.\n\u003e\n\u003e\n\u003e On Tue, Jun 18, 2013 at 7:45 PM, Turkey Breast \u003cturkeybreast at yahoo.com\u003ewrote:\n\u003e\n\u003e See this BIP. I'm not sure if this is a bug or what, but it would be good\n\u003e if messages always had a fixed number of fields per protocol version.\n\u003e\n\u003e https://en.bitcoin.it/wiki/BIP_0060#Code_Updates\n\u003e\n\u003e This BIP details everything that needs to be done and proposes a protocol\n\u003e upgrade.\n\u003e\n\u003e\n\u003e ------------------------------------------------------------------------------\n\u003e This SF.net \u003chttp://sf.net/\u003e email is sponsored by Windows:\n\u003e\n\u003e Build for Windows Store.\n\u003e\n\u003e http://p.sf.net/sfu/windows-dev2dev\n\u003e _______________________________________________\n\u003e Bitcoin-development mailing list\n\u003e Bitcoin-development at lists.sourceforge.net\n\u003e https://lists.sourceforge.net/lists/listinfo/bitcoin-development\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e ------------------------------------------------------------------------------\n\u003e This SF.net email is sponsored by Windows:\n\u003e\n\u003e Build for Windows Store.\n\u003e\n\u003e http://p.sf.net/sfu/windows-dev2dev\n\u003e _______________________________________________\n\u003e Bitcoin-development mailing list\n\u003e Bitcoin-development at lists.sourceforge.net\n\u003e https://lists.sourceforge.net/lists/listinfo/bitcoin-development\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e ------------------------------------------------------------------------------\n\u003e This SF.net email is sponsored by Windows:\n\u003e\n\u003e Build for Windows Store.\n\u003e\n\u003e http://p.sf.net/sfu/windows-dev2dev\n\u003e _______________________________________________\n\u003e Bitcoin-development mailing list\n\u003e Bitcoin-development at lists.sourceforge.net\n\u003e https://lists.sourceforge.net/lists/listinfo/bitcoin-development\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e\n\u003e ------------------------------------------------------------------------------\n\u003e This SF.net email is sponsored by Windows:\n\u003e\n\u003e Build for Windows Store.\n\u003e\n\u003e http://p.sf.net/sfu/windows-dev2dev\n\u003e _______________________________________________\n\u003e Bitcoin-development mailing list\n\u003e Bitcoin-development at lists.sourceforge.net\n\u003e https://lists.sourceforge.net/lists/listinfo/bitcoin-development\n\u003e\n\u003e\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20130620/0683b160/attachment.html\u003e",
"sig": "857d36e7191b018bb8d51151fe32db913e98060007137b803d5effddc5678d4b40a0168595e87bce50b74b3c5c559264efcad0e5bcfd1ada4f42d29373c679ba"
}