đź“… Original date posted:2019-07-27
đź“ť Original message:This conversation went off the rails somewhat. I don't think there's any immediate risk of NODE_BLOOM peers being unavailable. This is a defaults change, not a removal of the code to serve BIP 37 peers (nor would I suggest removing said code while people still want to use them - the maintenance burden isn't much). Looking at historical upgrade cycles, ignoring any other factors, there will be a large number of nodes serving NODE_BLOOM for many years.
Even more importantly, if you need them, run a node or two. As long as no one is exploiting the issues with them such a node isn't *too* expensive. Or don't, I guarantee you chainanalysis or some competitor of theirs will very very happily serve bloom-filtered clients as long as such clients want to deanonymize themselves. We already see a plurality of nodes on the network are clearly not run-of-the-mill Core nodes, many of which are likely deanonimization efforts.
In some cases BIP 137 is a replacement, in some cases, indeed, it is not. I agree at a protocol level we shouldn't be passing judgement about how users wish to interact with the Bitcoin system (aside from not putting our own, personal, effort into building such things) but that isn't what's happening here. This is an important DoS fix for the average node, and I don't really understand the argument that this is going to break existing BIP 37 wallets, but if it makes you feel any better I can run some beefy BIP 37 nodes.
Matt
> On Jul 26, 2019, at 06:04, Jonas Schnelli via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>
>> 1) It causes way too much traffic for mobile users, and likely even too
>> much traffic for fixed lines in not so developed parts of the world.
>
> Yes. It causes more traffic than BIP37.
> Basic block filters for current last ~7 days (1008 blocks) are about 19MB (just the filters).
> On top, you will probably fetch a handful of irrelevant blocks due to the FPs and due to true relevant txns.
> A over-the-thumb estimation: ~25MB per week of catch-up.
> If you where offline for a month: ~108MB
>
> Thats certainly more then BIP37 BF (measured 1.6MB total traffic with android schildbach wallet restore blockchain for 8 week [7 weeks headers, 1week merkleblocks]).
>
> But lets look at it like this: for an additional, say 25MB per week (maybe a bit more), you get the ability to filter blocks without depending on serving peers who may compromise your financial privacy.
> Also, if you keep the filters, further rescans do consume the same or less bandwidth than BF BIP37.
> In other words: you have the chance to potentially increase privacy by consuming bandwidth in the range of a single audio podcast per week.
>
> I would say the job of protocol developers is protect users privacy where it’s possible (as a default).
> It’s probably a debatable point wether 25MB per week of traffic is worth a potential increase in privacy, though I absolutely think 25MB/week is an acceptable tradeoff.
> Saving traffic is possible by using BIP37 or stratum/electrum… but developers should make sure users are __warned about the consequences__!
>
> Additionally, it looks like, peer operators are not endless being willing to serve – for free – a CPU/disk intense service with no benefits for the network. I would question wether a decentralised form of BIP37 is sustainable in the long run (if SPV wallet provider bootstrap a net range of NODE_BLOOM peers to make it more reliable on the network would be snake-oil).
>
>
>>
>> 2) It filters blocks only. It doesn't address unconfirmed transactions.
>
> Well, unconfirmed transaction are uncertain for various reasons.
>
> BIP158 won't allow you to filter the mempool.
> But as soon as you are connected to the network, you may fetch tx with inv/getdata and pick out the relevant ones (causes also traffic).
> Unclear and probably impossible with the current BIP158 specs to fetch transactions that are not in active relay and are not in a block (mempool txns, at least this is true with the current observed relay tactics).
>
>
>> 3) Afaik, it enforces/encourages address re-use. This stems from the
>> fact that the server decides on the filter and in particular on the
>> false positive rate. On wallets with many addresses, a hardcoded filter
>> will be too blurry and thus each block will be matched. So wallets that
>> follow the "one address per incoming payment" pattern (e.g. HD wallets)
>> at some point will be forced to wrap their key chains back to the
>> beginning. If I'm wrong on this one please let me know.
>
> I’m probably the wrong guy to ask (haven’t made the numbers) but last time I rescanned a Core wallet (in my dev branch) with block filters (and a Core wallet has >2000 addresses by default) it fetched a low and acceptable amount of false positive blocks.
> (Maybe someone who made the numbers step in here.)
>
> Though, large wallets – AFAIK – also operate badly with BIP37.
>
>>
>> 4) The filters are not yet committed to the blockchain. Until that
>> happens we'd have to trust a server to provide correct filters.
>
> I wouldn’t say so. It’s on a similar level than BIP37.
> BIP37 is not – and can not – be committed to the blockchain.
> You fully trust the peer that it won’t…
> a) create fake unconfirmed transactions (would be the same if a BIP158 wallet would show you unconfirmed transaction)
> b) lies by omission (you will miss relevant transactions, eventually swipe your wallet and loose coins)
>
> IMO, the point b) is true for BIP37 and BIP158 (as long as not commited).
> In both cases, you can reduce the trust by comparing between peers / filter-providers.
>
> b) is conceptually solvable with a soft-fork (commitment) in BIP158 (not with BIP37).
>
> Additionally, block-filters will, very likely, be useful for other features (load/rescan an [old] wallet on a prune peer that has the filters constructed).
>
>
>
> There is probably no clear answer like „X is better than Y“.
>
> Personally I would like to see developers being more honest/transparent to users about the implications of the used filtering,... and giving them choices.
> Imagine a user can choose between „Electrum / BIP37 / BIP158“ depending on his needs for privacy and availability of bandwidth. Eventually also taking the future usage of this wallet (will he load old private keys, will he receive money daily, etc.) into account.
>
> Plus, full node hardware appliances that run at home (or in a trusted environment) are solving many of these issues plus adding a bunch of great features – if done right.
>
> /jonas
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190727/9c417a9f/attachment-0001.html>;
Matt Corallo [ARCHIVE] /
npub1e4…jxmcu
2023-06-07 18:19:37
in reply to nevent1q…lgp8
Author Public Key
npub1e46n428mcyfwznl7nlsf6d3s7rhlwm9x3cmkuqzt3emmdpadmkaqqjxmcuSeen on
wss://relay.noswhere.comPublished at
2023-06-07 18:19:37Event JSON
{
"id": "4e708e244134df046651216d17433de7facc52aad234ff656645eed4955e3f74",
"pubkey": "cd753aa8fbc112e14ffe9fe09d3630f0eff76ca68e376e004b8e77b687adddba",
"created_at": 1686161977,
"kind": 1,
"tags": [
[
"e",
"9984a06ec701a8f6ac2773e4a62741a4d9d15af90b1fce9ee03180b7fd819eec",
"",
"root"
],
[
"e",
"f4fc8756d63c252b0e193e17ab0214f4b5cc0bfa19af597a9a0dfaf8cdd97ba4",
"",
"reply"
],
[
"p",
"9a463e0fab8963b013698c15a0f2449d19c97f3b88458e5874095b5006df9a0c"
]
],
"content": "📅 Original date posted:2019-07-27\n📝 Original message:This conversation went off the rails somewhat. I don't think there's any immediate risk of NODE_BLOOM peers being unavailable. This is a defaults change, not a removal of the code to serve BIP 37 peers (nor would I suggest removing said code while people still want to use them - the maintenance burden isn't much). Looking at historical upgrade cycles, ignoring any other factors, there will be a large number of nodes serving NODE_BLOOM for many years.\n\nEven more importantly, if you need them, run a node or two. As long as no one is exploiting the issues with them such a node isn't *too* expensive. Or don't, I guarantee you chainanalysis or some competitor of theirs will very very happily serve bloom-filtered clients as long as such clients want to deanonymize themselves. We already see a plurality of nodes on the network are clearly not run-of-the-mill Core nodes, many of which are likely deanonimization efforts.\n\nIn some cases BIP 137 is a replacement, in some cases, indeed, it is not. I agree at a protocol level we shouldn't be passing judgement about how users wish to interact with the Bitcoin system (aside from not putting our own, personal, effort into building such things) but that isn't what's happening here. This is an important DoS fix for the average node, and I don't really understand the argument that this is going to break existing BIP 37 wallets, but if it makes you feel any better I can run some beefy BIP 37 nodes.\n\nMatt\n\n\u003e On Jul 26, 2019, at 06:04, Jonas Schnelli via bitcoin-dev \u003cbitcoin-dev at lists.linuxfoundation.org\u003e wrote:\n\u003e \n\u003e \n\u003e\u003e 1) It causes way too much traffic for mobile users, and likely even too\n\u003e\u003e much traffic for fixed lines in not so developed parts of the world.\n\u003e \n\u003e Yes. It causes more traffic than BIP37.\n\u003e Basic block filters for current last ~7 days (1008 blocks) are about 19MB (just the filters).\n\u003e On top, you will probably fetch a handful of irrelevant blocks due to the FPs and due to true relevant txns.\n\u003e A over-the-thumb estimation: ~25MB per week of catch-up.\n\u003e If you where offline for a month: ~108MB\n\u003e \n\u003e Thats certainly more then BIP37 BF (measured 1.6MB total traffic with android schildbach wallet restore blockchain for 8 week [7 weeks headers, 1week merkleblocks]).\n\u003e \n\u003e But lets look at it like this: for an additional, say 25MB per week (maybe a bit more), you get the ability to filter blocks without depending on serving peers who may compromise your financial privacy.\n\u003e Also, if you keep the filters, further rescans do consume the same or less bandwidth than BF BIP37.\n\u003e In other words: you have the chance to potentially increase privacy by consuming bandwidth in the range of a single audio podcast per week.\n\u003e \n\u003e I would say the job of protocol developers is protect users privacy where it’s possible (as a default).\n\u003e It’s probably a debatable point wether 25MB per week of traffic is worth a potential increase in privacy, though I absolutely think 25MB/week is an acceptable tradeoff.\n\u003e Saving traffic is possible by using BIP37 or stratum/electrum… but developers should make sure users are __warned about the consequences__!\n\u003e \n\u003e Additionally, it looks like, peer operators are not endless being willing to serve – for free – a CPU/disk intense service with no benefits for the network. I would question wether a decentralised form of BIP37 is sustainable in the long run (if SPV wallet provider bootstrap a net range of NODE_BLOOM peers to make it more reliable on the network would be snake-oil).\n\u003e \n\u003e \n\u003e\u003e \n\u003e\u003e 2) It filters blocks only. It doesn't address unconfirmed transactions.\n\u003e \n\u003e Well, unconfirmed transaction are uncertain for various reasons.\n\u003e \n\u003e BIP158 won't allow you to filter the mempool.\n\u003e But as soon as you are connected to the network, you may fetch tx with inv/getdata and pick out the relevant ones (causes also traffic).\n\u003e Unclear and probably impossible with the current BIP158 specs to fetch transactions that are not in active relay and are not in a block (mempool txns, at least this is true with the current observed relay tactics).\n\u003e \n\u003e \n\u003e\u003e 3) Afaik, it enforces/encourages address re-use. This stems from the\n\u003e\u003e fact that the server decides on the filter and in particular on the\n\u003e\u003e false positive rate. On wallets with many addresses, a hardcoded filter\n\u003e\u003e will be too blurry and thus each block will be matched. So wallets that\n\u003e\u003e follow the \"one address per incoming payment\" pattern (e.g. HD wallets)\n\u003e\u003e at some point will be forced to wrap their key chains back to the\n\u003e\u003e beginning. If I'm wrong on this one please let me know.\n\u003e \n\u003e I’m probably the wrong guy to ask (haven’t made the numbers) but last time I rescanned a Core wallet (in my dev branch) with block filters (and a Core wallet has \u003e2000 addresses by default) it fetched a low and acceptable amount of false positive blocks.\n\u003e (Maybe someone who made the numbers step in here.)\n\u003e \n\u003e Though, large wallets – AFAIK – also operate badly with BIP37.\n\u003e \n\u003e\u003e \n\u003e\u003e 4) The filters are not yet committed to the blockchain. Until that\n\u003e\u003e happens we'd have to trust a server to provide correct filters.\n\u003e \n\u003e I wouldn’t say so. It’s on a similar level than BIP37.\n\u003e BIP37 is not – and can not – be committed to the blockchain.\n\u003e You fully trust the peer that it won’t…\n\u003e a) create fake unconfirmed transactions (would be the same if a BIP158 wallet would show you unconfirmed transaction)\n\u003e b) lies by omission (you will miss relevant transactions, eventually swipe your wallet and loose coins)\n\u003e \n\u003e IMO, the point b) is true for BIP37 and BIP158 (as long as not commited).\n\u003e In both cases, you can reduce the trust by comparing between peers / filter-providers.\n\u003e \n\u003e b) is conceptually solvable with a soft-fork (commitment) in BIP158 (not with BIP37).\n\u003e \n\u003e Additionally, block-filters will, very likely, be useful for other features (load/rescan an [old] wallet on a prune peer that has the filters constructed).\n\u003e \n\u003e \n\u003e \n\u003e There is probably no clear answer like „X is better than Y“.\n\u003e \n\u003e Personally I would like to see developers being more honest/transparent to users about the implications of the used filtering,... and giving them choices.\n\u003e Imagine a user can choose between „Electrum / BIP37 / BIP158“ depending on his needs for privacy and availability of bandwidth. Eventually also taking the future usage of this wallet (will he load old private keys, will he receive money daily, etc.) into account.\n\u003e \n\u003e Plus, full node hardware appliances that run at home (or in a trusted environment) are solving many of these issues plus adding a bunch of great features – if done right.\n\u003e \n\u003e /jonas\n\u003e _______________________________________________\n\u003e bitcoin-dev mailing list\n\u003e bitcoin-dev at lists.linuxfoundation.org\n\u003e https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190727/9c417a9f/attachment-0001.html\u003e",
"sig": "431d6cdea4ea19a66a2c71282437f48a51f128d99d880dd98fe987bfbf6d7f4e4541cda9ec559941e86f555ad3f42561c5efc739fb4aaf114628e439ed8f7cf9"
}