📅 Original date posted:2018-02-07
📝 Original message:
Hi ZmnSCPxj and Laolu,
> Indeed, the existence of per-hop fees (`fee_base_msat`) means, splitting
the
> payment over multiple flows will be, very likely, more expensive,
compared to
> using a single flow.
As Laolu pointed out, we have yet to see how fees evolve on mainnet or what
will
emerge as a sane, default fee schedules. I agree that if the same
proportional
fee is used across all partial payments, then it could certainly be more
expensive.
However, it could also be the case that you were paying a needlessly high
proportional fee to begin with, because paths of sufficient capacity to the
destination were scarce. In an AMP world, there will be an abundance of
channels
that can route small, partial payments, which may itself drive down the
competitive fee rate for smaller payments. Just a hypothesis, we shall see
where
supply meets demand!
At the end of the day, the user can always fall back to regular payment if
they
expect to end up paying more fees using an AMP.
> (If we want to support multiple routes converging to an intermediate node,
> then continue routing to a different final node after routes have merged
(i.e.
> A->B->C->D, and A->E->C->D, with the payment being merged by C, who
forwards
> the combination to D), then we need to follow the current hop data
format, but
> I think supporting AMP at final payees is actually enough...
I think this is an interesting idea, sounds maybe like a
recursive/hierarchical
AMP? The ability to merge the payments seems like it would result in a
decent privacy
leak, as I believe an intermediary would have enough evidence to prove that
two
payments were merged/correlated. Simple traffic analysis would also reveal a
discrepancy in the number of incoming and outgoing packets, and possibly
other
observable differences in routing (some) AMPs vs regular payments.
FWIW the current proposal allows the paths of partial payments to overlap,
in such a scenario C would just forward the HTLCs independently. One could
send
them all along the same path if they desired! I'm assuming the intent here
is to
try and reduce total fees?
Minor correction^2:
> This should actually be (H(s_0 || s_1 || ...), s_i).
This assumes the receiver knows the indexes of each share. Without this
knowledge they would have to brute force all orderings to check the
fingerprint.
To maintain order invariance on the receiving end, I would propose sending
(0, s_i) for the first n-1 partial payments, and then (n, s_i) on the final
one.
As in the description of the basic AMP scheme, the receiver maintains a
persistent count of how many partial payments have been received for ID. If
the
receiver does not get the last payment last, the receiver just waits until
all n
have been received before deciding that its reconstructed value is BP.
The receiver can verify they've received the correct BP and n by rederiving
the
partial preimages r_i = H(BP || i) and checking that there are n outstanding
payments, one for each h_i = H(r_i). This also saves the receiving node n
additional hash invocations.
-Conner
On Tue, Feb 6, 2018 at 4:04 PM Olaoluwa Osuntokun <laolu32 at gmail.com> wrote:
> Hi ZmnSCPxj,
>
> > This is excellent work!
>
> Thanks!
>
> > I think, a `globalfeatures` odd bit could be used for this. As it is
> > end-ot-end, `localfeatures` is not appropriate.
>
> Yep, it would need to be a global feature bit. In the case that we're
> sending to a destination which isn't publicly advertised, then perhaps an
> extension to BOLT-11 could be made to signal receiver support.
>
> > I believe, currently, fees have not this super-linear component
>
> Yep they don't. Arguably, we should also have a component that scales
> according to the proposed CLTV value of the outgoing HTLC. At Scaling
> Bitcoin Stanford, Aviv Zohar gave a talked titled "How to Charge Lightning"
> where the authors analyzed the possible evolution of fees on the network
> (and also suggested adding this super-linear component to extend the
> lifetime of channels). However, the talk itself focused on a very simple
> "mega super duper hub" topology. Towards the end he alluded to a
> forthcoming
> paper that had more comprehensive analysis of more complex topologies. I
> look forward to the publication of their finalized work.
>
> > Indeed, the existence of per-hop fees (`fee_base_msat`) means, splitting
> > the payment over multiple flows will be, very likely, more expensive,
> > compared to using a single flow.
>
> Well it's still to be seen how the fee structure on mainnet emerges once
> the
> network is still fully bootstrapped. AFAIK, most running on mainnet atm are
> using the default fee schedules for their respective implementations. For
> example, the default fee_base_msat for lnd is 1000 msat (1 satoshi).
>
> > I believe the `realm` byte is intended for this.
>
> The realm byte is meant to signal "forward this to the dogecoin channel".
> ATM, we just default to 0 as "Bitcoin". However, the byte itself only
> really
> need significance between the sender and the intermediate node. So there
> isn't necessarily pressure to have a globally synchronized set of realm
> bytes.
>
> > Thus, you can route over nodes that are unaware of AMP, and only provide
> > an AMP realm byte to the destination node, who, is able to reconstruct
> this
> > your AMP data as per your algorithm.
>
> Yes, the intermediate nodes don't need to be aware of the end-to-end
> protocol. For the final hop, there are actually 53 free bytes (before one
> needs to signal the existence of EOBs):
>
> * 1 byte realm
> * 8 bytes next addr (all zeroes to signal final dest)
> * 32 bytes hmac (also all zeroes for the final dest)
> * 12 bytes padding
>
> So any combo of these bytes can be used to signal more advanced protocols
> to
> the final destination.
>
>
> A correction from the prior email description:
>
> > We can further modify our usage of the per-hop payloads to send
> > (H(BP), s_i) to consume most of the EOB sent from sender to receiver.
>
> This should actually be (H(s_0 || s_1 || ...), s_i). So we still allow them
> to check this finger print to see if they have all the final shares, but
> don't allow them to preemptively pull all the payments.
>
>
> -- Laolu
>
>
> On Mon, Feb 5, 2018 at 11:12 PM ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:
>
>> Good morning Laolu,
>>
>> This is excellent work!
>>
>> Some minor comments...
>>
>>
>> (Atomic Multi-path Payments). It can be experimented with on Lightning
>> *today* with the addition of a new feature bit to gate this new
>> feature. The beauty of the scheme is that it requires no fundamental
>> changes
>> to the protocol as is now, as the negotiation is strictly *end-to-end*
>> between sender and receiver.
>>
>>
>> I think, a `globalfeatures` odd bit could be used for this. As it is
>> end-ot-end, `localfeatures` is not appropriate.
>>
>> - Potential fee savings for larger payments, contingent on there being a
>> super-linear component to routed fees. It's possible that with
>> modifications to the fee schedule, it's actually *cheaper* to send
>> payments over multiple flows rather than one giant flow.
>>
>>
>> I believe, currently, fees have not this super-linear component. Indeed,
>> the existence of per-hop fees (`fee_base_msat`) means, splitting the
>> payment over multiple flows will be, very likely, more expensive, compared
>> to using a single flow. Tiny roundoffs in computing the proportional fees
>> (`fee_proportional_millionths`) may make smaller flows give a slight fee
>> advantage, but I think the multiplication of per-hop fees will dominate.
>>
>>
>> - Using smaller payments increases the set of possible paths a partial
>> payment could have taken, which reduces the effectiveness of static
>> analysis techniques involving channel capacities and the plaintext
>> values being forwarded.
>>
>>
>> Strongly agree!
>>
>>
>> In order to include the three tuple within the per-hop payload for the
>> final
>> destination, we repurpose the _first_ byte of the un-used padding bytes in
>> the payload to signal version 0x01 of the AMP protocol (note this is a PoC
>> outline, we would need to standardize signalling of these 12 bytes to
>> support other protocols).
>>
>>
>> I believe the `realm` byte is intended for this. Intermediate nodes do
>> not need to understand realm bytes that are understood by other nodes in
>> the route, including the realm bytes understood by the final destination,
>> as intermediate nodes cannot, indeed, read the hop data of other nodes.
>> Thus, you can route over nodes that are unaware of AMP, and only provide an
>> AMP realm byte to the destination node, who, is able to reconstruct this
>> your AMP data as per your algorithm.
>>
>> Indeed, the `realm` byte controls the interpretation of the rest of the
>> 65-byte packet. If you define, instead, a separate `realm` that is
>> understood by the destination node, you can redefine the entire 64 bytes of
>> the final hop data as you wish.
>>
>> If we support AMP only at final payees, we can completely redefine the 64
>> bytes in the final hop data for the new AMP `realm`, and not consume the
>> next hop (which would reduce route length by 1).
>>
>> (If we want to support multiple routes converging to an intermediate
>> node, then continue routing to a different final node after routes have
>> merged (i.e. A->B->C->D, and A->E->C->D, with the payment being merged by
>> C, who forwards the combination to D), then we need to follow the current
>> hop data format, but I think supporting AMP at final payees is actually
>> enough... AMP at intermediate nodes might not be used often enough by
>> senders for it to matter, as taking advantage of that seems more complex
>> than just asking your routing algo to provide you multiple routes to a
>> destination, which you are probably already doing)
>>
>> ----
>>
>> Overall, good work I think.
>>
>> Regards,
>> ZmnSCPxj
>>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180207/686a561b/attachment.html>;
Conner Fromknecht [ARCHIVE] /
npub1za…uua2a
2023-06-09 12:48:58
in reply to nevent1q…7pfg
Author Public Key
npub1za0a9afyj7um5feva0d5xmhfsah3zxm252hna2duq0numa952frqvuua2aPublished at
2023-06-09 12:48:58Event JSON
{
"id": "35c4fd7429e92d684d35115dff5dc210371d8c23bc1ec5e32b0c275865373f8b",
"pubkey": "175fd2f52497b9ba272cebdb436ee9876f111b6aa2af3ea9bc03e7cdf4b45246",
"created_at": 1686314938,
"kind": 1,
"tags": [
[
"e",
"51bc92b7fa9a7d9c09bb73c2362ca09a737a241a77a68cd95503c148136969eb",
"",
"root"
],
[
"e",
"661f4aa02c12be4948420e920404a3f778b10c543105071f428ebe9c198f6aa7",
"",
"reply"
],
[
"p",
"2df3fc2660459521b852c995d4fc1a93938389a5e085677d0ebb33ef92cc5476"
]
],
"content": "📅 Original date posted:2018-02-07\n📝 Original message:\nHi ZmnSCPxj and Laolu,\n\n\u003e Indeed, the existence of per-hop fees (`fee_base_msat`) means, splitting\nthe\n\u003e payment over multiple flows will be, very likely, more expensive,\ncompared to\n\u003e using a single flow.\n\nAs Laolu pointed out, we have yet to see how fees evolve on mainnet or what\nwill\nemerge as a sane, default fee schedules. I agree that if the same\nproportional\nfee is used across all partial payments, then it could certainly be more\nexpensive.\n\nHowever, it could also be the case that you were paying a needlessly high\nproportional fee to begin with, because paths of sufficient capacity to the\ndestination were scarce. In an AMP world, there will be an abundance of\nchannels\nthat can route small, partial payments, which may itself drive down the\ncompetitive fee rate for smaller payments. Just a hypothesis, we shall see\nwhere\nsupply meets demand!\n\nAt the end of the day, the user can always fall back to regular payment if\nthey\nexpect to end up paying more fees using an AMP.\n\n\u003e (If we want to support multiple routes converging to an intermediate node,\n\u003e then continue routing to a different final node after routes have merged\n(i.e.\n\u003e A-\u003eB-\u003eC-\u003eD, and A-\u003eE-\u003eC-\u003eD, with the payment being merged by C, who\nforwards\n\u003e the combination to D), then we need to follow the current hop data\nformat, but\n\u003e I think supporting AMP at final payees is actually enough...\n\nI think this is an interesting idea, sounds maybe like a\nrecursive/hierarchical\nAMP? The ability to merge the payments seems like it would result in a\ndecent privacy\nleak, as I believe an intermediary would have enough evidence to prove that\ntwo\npayments were merged/correlated. Simple traffic analysis would also reveal a\ndiscrepancy in the number of incoming and outgoing packets, and possibly\nother\nobservable differences in routing (some) AMPs vs regular payments.\n\nFWIW the current proposal allows the paths of partial payments to overlap,\nin such a scenario C would just forward the HTLCs independently. One could\nsend\nthem all along the same path if they desired! I'm assuming the intent here\nis to\ntry and reduce total fees?\n\nMinor correction^2:\n\n\u003e This should actually be (H(s_0 || s_1 || ...), s_i).\n\nThis assumes the receiver knows the indexes of each share. Without this\nknowledge they would have to brute force all orderings to check the\nfingerprint.\n\nTo maintain order invariance on the receiving end, I would propose sending\n(0, s_i) for the first n-1 partial payments, and then (n, s_i) on the final\none.\nAs in the description of the basic AMP scheme, the receiver maintains a\npersistent count of how many partial payments have been received for ID. If\nthe\nreceiver does not get the last payment last, the receiver just waits until\nall n\nhave been received before deciding that its reconstructed value is BP.\n\nThe receiver can verify they've received the correct BP and n by rederiving\nthe\npartial preimages r_i = H(BP || i) and checking that there are n outstanding\npayments, one for each h_i = H(r_i). This also saves the receiving node n\nadditional hash invocations.\n\n-Conner\n\nOn Tue, Feb 6, 2018 at 4:04 PM Olaoluwa Osuntokun \u003claolu32 at gmail.com\u003e wrote:\n\n\u003e Hi ZmnSCPxj,\n\u003e\n\u003e \u003e This is excellent work!\n\u003e\n\u003e Thanks!\n\u003e\n\u003e \u003e I think, a `globalfeatures` odd bit could be used for this. As it is\n\u003e \u003e end-ot-end, `localfeatures` is not appropriate.\n\u003e\n\u003e Yep, it would need to be a global feature bit. In the case that we're\n\u003e sending to a destination which isn't publicly advertised, then perhaps an\n\u003e extension to BOLT-11 could be made to signal receiver support.\n\u003e\n\u003e \u003e I believe, currently, fees have not this super-linear component\n\u003e\n\u003e Yep they don't. Arguably, we should also have a component that scales\n\u003e according to the proposed CLTV value of the outgoing HTLC. At Scaling\n\u003e Bitcoin Stanford, Aviv Zohar gave a talked titled \"How to Charge Lightning\"\n\u003e where the authors analyzed the possible evolution of fees on the network\n\u003e (and also suggested adding this super-linear component to extend the\n\u003e lifetime of channels). However, the talk itself focused on a very simple\n\u003e \"mega super duper hub\" topology. Towards the end he alluded to a\n\u003e forthcoming\n\u003e paper that had more comprehensive analysis of more complex topologies. I\n\u003e look forward to the publication of their finalized work.\n\u003e\n\u003e \u003e Indeed, the existence of per-hop fees (`fee_base_msat`) means, splitting\n\u003e \u003e the payment over multiple flows will be, very likely, more expensive,\n\u003e \u003e compared to using a single flow.\n\u003e\n\u003e Well it's still to be seen how the fee structure on mainnet emerges once\n\u003e the\n\u003e network is still fully bootstrapped. AFAIK, most running on mainnet atm are\n\u003e using the default fee schedules for their respective implementations. For\n\u003e example, the default fee_base_msat for lnd is 1000 msat (1 satoshi).\n\u003e\n\u003e \u003e I believe the `realm` byte is intended for this.\n\u003e\n\u003e The realm byte is meant to signal \"forward this to the dogecoin channel\".\n\u003e ATM, we just default to 0 as \"Bitcoin\". However, the byte itself only\n\u003e really\n\u003e need significance between the sender and the intermediate node. So there\n\u003e isn't necessarily pressure to have a globally synchronized set of realm\n\u003e bytes.\n\u003e\n\u003e \u003e Thus, you can route over nodes that are unaware of AMP, and only provide\n\u003e \u003e an AMP realm byte to the destination node, who, is able to reconstruct\n\u003e this\n\u003e \u003e your AMP data as per your algorithm.\n\u003e\n\u003e Yes, the intermediate nodes don't need to be aware of the end-to-end\n\u003e protocol. For the final hop, there are actually 53 free bytes (before one\n\u003e needs to signal the existence of EOBs):\n\u003e\n\u003e * 1 byte realm\n\u003e * 8 bytes next addr (all zeroes to signal final dest)\n\u003e * 32 bytes hmac (also all zeroes for the final dest)\n\u003e * 12 bytes padding\n\u003e\n\u003e So any combo of these bytes can be used to signal more advanced protocols\n\u003e to\n\u003e the final destination.\n\u003e\n\u003e\n\u003e A correction from the prior email description:\n\u003e\n\u003e \u003e We can further modify our usage of the per-hop payloads to send\n\u003e \u003e (H(BP), s_i) to consume most of the EOB sent from sender to receiver.\n\u003e\n\u003e This should actually be (H(s_0 || s_1 || ...), s_i). So we still allow them\n\u003e to check this finger print to see if they have all the final shares, but\n\u003e don't allow them to preemptively pull all the payments.\n\u003e\n\u003e\n\u003e -- Laolu\n\u003e\n\u003e\n\u003e On Mon, Feb 5, 2018 at 11:12 PM ZmnSCPxj \u003cZmnSCPxj at protonmail.com\u003e wrote:\n\u003e\n\u003e\u003e Good morning Laolu,\n\u003e\u003e\n\u003e\u003e This is excellent work!\n\u003e\u003e\n\u003e\u003e Some minor comments...\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e (Atomic Multi-path Payments). It can be experimented with on Lightning\n\u003e\u003e *today* with the addition of a new feature bit to gate this new\n\u003e\u003e feature. The beauty of the scheme is that it requires no fundamental\n\u003e\u003e changes\n\u003e\u003e to the protocol as is now, as the negotiation is strictly *end-to-end*\n\u003e\u003e between sender and receiver.\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e I think, a `globalfeatures` odd bit could be used for this. As it is\n\u003e\u003e end-ot-end, `localfeatures` is not appropriate.\n\u003e\u003e\n\u003e\u003e - Potential fee savings for larger payments, contingent on there being a\n\u003e\u003e super-linear component to routed fees. It's possible that with\n\u003e\u003e modifications to the fee schedule, it's actually *cheaper* to send\n\u003e\u003e payments over multiple flows rather than one giant flow.\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e I believe, currently, fees have not this super-linear component. Indeed,\n\u003e\u003e the existence of per-hop fees (`fee_base_msat`) means, splitting the\n\u003e\u003e payment over multiple flows will be, very likely, more expensive, compared\n\u003e\u003e to using a single flow. Tiny roundoffs in computing the proportional fees\n\u003e\u003e (`fee_proportional_millionths`) may make smaller flows give a slight fee\n\u003e\u003e advantage, but I think the multiplication of per-hop fees will dominate.\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e - Using smaller payments increases the set of possible paths a partial\n\u003e\u003e payment could have taken, which reduces the effectiveness of static\n\u003e\u003e analysis techniques involving channel capacities and the plaintext\n\u003e\u003e values being forwarded.\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e Strongly agree!\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e In order to include the three tuple within the per-hop payload for the\n\u003e\u003e final\n\u003e\u003e destination, we repurpose the _first_ byte of the un-used padding bytes in\n\u003e\u003e the payload to signal version 0x01 of the AMP protocol (note this is a PoC\n\u003e\u003e outline, we would need to standardize signalling of these 12 bytes to\n\u003e\u003e support other protocols).\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e I believe the `realm` byte is intended for this. Intermediate nodes do\n\u003e\u003e not need to understand realm bytes that are understood by other nodes in\n\u003e\u003e the route, including the realm bytes understood by the final destination,\n\u003e\u003e as intermediate nodes cannot, indeed, read the hop data of other nodes.\n\u003e\u003e Thus, you can route over nodes that are unaware of AMP, and only provide an\n\u003e\u003e AMP realm byte to the destination node, who, is able to reconstruct this\n\u003e\u003e your AMP data as per your algorithm.\n\u003e\u003e\n\u003e\u003e Indeed, the `realm` byte controls the interpretation of the rest of the\n\u003e\u003e 65-byte packet. If you define, instead, a separate `realm` that is\n\u003e\u003e understood by the destination node, you can redefine the entire 64 bytes of\n\u003e\u003e the final hop data as you wish.\n\u003e\u003e\n\u003e\u003e If we support AMP only at final payees, we can completely redefine the 64\n\u003e\u003e bytes in the final hop data for the new AMP `realm`, and not consume the\n\u003e\u003e next hop (which would reduce route length by 1).\n\u003e\u003e\n\u003e\u003e (If we want to support multiple routes converging to an intermediate\n\u003e\u003e node, then continue routing to a different final node after routes have\n\u003e\u003e merged (i.e. A-\u003eB-\u003eC-\u003eD, and A-\u003eE-\u003eC-\u003eD, with the payment being merged by\n\u003e\u003e C, who forwards the combination to D), then we need to follow the current\n\u003e\u003e hop data format, but I think supporting AMP at final payees is actually\n\u003e\u003e enough... AMP at intermediate nodes might not be used often enough by\n\u003e\u003e senders for it to matter, as taking advantage of that seems more complex\n\u003e\u003e than just asking your routing algo to provide you multiple routes to a\n\u003e\u003e destination, which you are probably already doing)\n\u003e\u003e\n\u003e\u003e ----\n\u003e\u003e\n\u003e\u003e Overall, good work I think.\n\u003e\u003e\n\u003e\u003e Regards,\n\u003e\u003e ZmnSCPxj\n\u003e\u003e\n\u003e _______________________________________________\n\u003e Lightning-dev mailing list\n\u003e Lightning-dev at lists.linuxfoundation.org\n\u003e https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev\n\u003e\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180207/686a561b/attachment.html\u003e",
"sig": "7b0d4bb24da0e6fd0fd6fef1ce27dddb83241fec66ef33848530cec0c69f736ad5b5bc81fd2397f158021daa60c09cefd741414a6cac696d79b5742252a7f638"
}