đź“… Original date posted:2019-01-08
đź“ť Original message:
Fabrice Drouin <fabrice.drouin at acinq.fr> writes:
> Follow-up: here's more detailed info on the data I collected and
> potential savings we could achieve:
>
> I made hourly routing table backups for 12 days, and collected routing
> information for 17 000 channel ids.
>
> There are 130 000 different channel updates :on average each channel
> has been updated 8 times. Here, “different” means that at least the
> timestamp has changed, and a node would have queried this channel
> update during its syncing process.
Side note: some implementations are also sending out updates with the
*same* timestamp. This is not allowed...
> But only 18 000 pairs of channel updates carry actual fee and/or HTLC
> value change. 85% of the time, we just queried information that we
> already had!
Note that this can happen in two legitimate cases:
1. The weekly refresh of channel_update.
2. A node updated too fast (A->B->A) and the ->A update caught up with the
->B update.
Fortunately, this seems fairly easy to handle: discard the newer
duplicate (unless > 1 week old). For future more advanced
reconstruction schemes (eg. INV or minisketch), we could remember the
latest timestamp of the duplicate, so we can avoid requesting it again.
> Adding a basic checksum (4 bytes for example) that covers fees and
> HTLC min/max value to our channel range queries would be a significant
> improvement and I will add this the open BOLT 1.1 proposal to extend
> queries with timestamps.
>
> I also think that such a checksum could also be used
> - in “inventory” based gossip messages
> - in set reconciliation schemes: we could reconcile [channel id |
> timestamp | checksum] first
I think this is overkill?
Thanks,
Rusty.
Rusty Russell [ARCHIVE] /
npub1zw…hkhpx
2023-06-09 12:53:51
in reply to nevent1q…0euw
Author Public Key
npub1zw7cc8z78v6s3grujfvcv3ckpvg6kr0w7nz9yzvwyglyg0qu5sjsqhkhpxPublished at
2023-06-09 12:53:51Event JSON
{
"id": "0be0c1e4270ac05a9c8cceac122e76a1723dc1b050b165449fb7e20f2fefc296",
"pubkey": "13bd8c1c5e3b3508a07c92598647160b11ab0deef4c452098e223e443c1ca425",
"created_at": 1686315231,
"kind": 1,
"tags": [
[
"e",
"fd0da5dbd5383b525edc98216d5094b180c1b831bf7af1f8df8ca35294a8c8fd",
"",
"root"
],
[
"e",
"36ac1dd9884dc55707393ea3a4f440e0eb882c225977c168dad52fba292f0d96",
"",
"reply"
],
[
"p",
"81c48ba46c211bc8fdb490d1ccfb03609c7ea090f8587ddca1c990676f09cfd3"
]
],
"content": "📅 Original date posted:2019-01-08\n📝 Original message:\nFabrice Drouin \u003cfabrice.drouin at acinq.fr\u003e writes:\n\u003e Follow-up: here's more detailed info on the data I collected and\n\u003e potential savings we could achieve:\n\u003e\n\u003e I made hourly routing table backups for 12 days, and collected routing\n\u003e information for 17 000 channel ids.\n\u003e\n\u003e There are 130 000 different channel updates :on average each channel\n\u003e has been updated 8 times. Here, “different” means that at least the\n\u003e timestamp has changed, and a node would have queried this channel\n\u003e update during its syncing process.\n\nSide note: some implementations are also sending out updates with the\n*same* timestamp. This is not allowed...\n\n\u003e But only 18 000 pairs of channel updates carry actual fee and/or HTLC\n\u003e value change. 85% of the time, we just queried information that we\n\u003e already had!\n\nNote that this can happen in two legitimate cases:\n1. The weekly refresh of channel_update.\n2. A node updated too fast (A-\u003eB-\u003eA) and the -\u003eA update caught up with the\n -\u003eB update.\n \nFortunately, this seems fairly easy to handle: discard the newer\nduplicate (unless \u003e 1 week old). For future more advanced\nreconstruction schemes (eg. INV or minisketch), we could remember the\nlatest timestamp of the duplicate, so we can avoid requesting it again.\n\n\u003e Adding a basic checksum (4 bytes for example) that covers fees and\n\u003e HTLC min/max value to our channel range queries would be a significant\n\u003e improvement and I will add this the open BOLT 1.1 proposal to extend\n\u003e queries with timestamps.\n\u003e\n\u003e I also think that such a checksum could also be used\n\u003e - in “inventory” based gossip messages\n\u003e - in set reconciliation schemes: we could reconcile [channel id |\n\u003e timestamp | checksum] first\n\nI think this is overkill?\n\nThanks,\nRusty.",
"sig": "14bdd2179a4038faa403da987ee6db3594eba0082b80b87c6949a24de7f74f16e912c24bb9b494e4ef7634e37e35af476c357da9e86a31f97a71ce573e43fd45"
}