📅 Original date posted:2023-02-14
📝 Original message:
Hi Joost,
> For a long time I've held the expectation that eventually payers on the
lightning network will become very strict about node performance. That they
will > require a routing node to operate flawlessly or else apply a hefty
penalty such as completely avoiding the node for an extended period of time
- multiple > weeks. The consequence of this is that routing nodes would
need to manage their liquidity meticulously because every failure
potentially has a large
> impact on future routing revenue.
I think the performance question depends on the type of payment flows
considered. If you're an
end-user sending a payment to your local Starbucks for coffee, here fast
payment sounds the end-goal.
If you're doing remittance payment, cheap fees might be favored, and in
function of those flows you're
probably not going to select the same "performant" routing nodes. I think
adding latency as a criteria for
pathfinding construction has already been mentioned in the past for LDK [0].
> I think movement in this direction is important to guarantee
competitiveness with centralised payment systems and their (at least
theoretical) ability to
> process a payment in the blink of an eye. A lightning wallet trying
multiple paths to find one that works doesn't help with this.
Or there is the direction to build forward-error-correction code on top of
MPP, like in traditional
networking [1]. The rough idea, you send more payment shards than the
requested sum, and then
you reveal the payment secrets to the receiver after an onion interactivity
round to finalize payment.
> A common argument against strict penalisation is that it would lead to
less efficient use of capital. Routing nodes would need to maintain pools of
> liquidity to guarantee successes all the time. My opinion on this is that
lightning is already enormously capital efficient at scale and that it is
worth
> sacrificing a slight part of that efficiency to also achieve the lowest
possible latency.
At the end of the day, we add more signal channels between HTLC senders and
the routing
nodes offering capital liquidity, if the signal mechanisms are efficient, I
think they should lead
to better allocation of the capital. So yes, I think more liquidity might
be used by routing nodes
to serve finely tailored HTLC requests by senders, however this liquidity
should be rewarded
by higher routing fees.
> This brings me to the actual subject of this post. Assuming strict
penalisation is good, it may still not be ideal to flip the switch from one
day to the other. > Routing nodes may not offer the required level of
service yet, causing senders to end up with no nodes to choose from.
> One option is to gradually increase the strength of the penalties, so
that routing nodes are given time to adapt to the new standards. This does
require > everyone to move along and leaves no space for cheap routing
nodes with less leeway in terms of liquidity.
I think if we have lessons to learn on policy rules design and deployment
on the base-layer
(the full-rbf saga), it's to be careful in the initial set of rules, and
how we ensure smooth
upgradeability, from one version to another. Otherwise the re-deployment
cost towards
the new version might incentive the old routing node to stay on the
non-optimal versions,
and as we have historical buckets in routing algorithms, or preference for
older channels,
this might lead the end-user to pay higher fees, than they could access to.
> Therefore I am proposing another way to go about it: extend the
`channel_update` field `channel_flags` with a new bit that the sender can
use to signal > `highly_available`.
> It's then up to payers to decide how to interpret this flag. One way
could be to prefer `highly_available` channels during pathfinding. But if
the routing
> node then returns a failure, a much stronger than normal penalty will be
applied. For routing nodes this creates an opportunity to attract more
traffic by > marking some channels as `highly_available`, but it also comes
with the responsibility to deliver.
This is where the open question lies to me - "highly available" can be
defined with multiple
senses, like fault-tolerance, latency processing, equilibrated liquidity.
And a routing node might
not be able to optimize its architecture for the same end-goal (e.g more
watchtower on remote
host probably increases the latency processing).
> Without shadow channels, it is impossible to guarantee liquidity up to
the channel capacity. It might make sense for senders to only assume high
> availability for amounts up to `htlc_maximum_msat`.
As a note, I think "senders assumption" should be well-documented,
otherwise there will be
performance discrepancies between node implementations or even versions.
E.g, an upgraded
sender penalizing a node for the lack of shadow/parallel channels
fulfilling HTLC amounts up to
`htlc_maximum_msat`.
> A variation on this scheme that requires no extension of `channel_update`
is to signal availability implicitly through routing fees. So the more
expensive > a channel is, the stronger the penalty that is applied on
failure will be. It seems less ideal though, because it
could disincentivize cheap but reliable
> channels on high traffic links.
> The effort required to implement some form of a `highly_available` flag
seem limited and it may help to get payment success rates up. Interested to
> hear your thoughts.
I think signal availability should be explicit rather than implicit. Even
if it's coming with more
gossip bandwidth data consumed. I would say for bandwidth performance
management, relying
on new gossip messages, where they can be filtered in function of the level
of services required
is interesting.
Best,
Antoine
[0] https://github.com/lightningdevkit/rust-lightning/issues/1647
[1] https://www.rfc-editor.org/rfc/rfc6363.html
Le lun. 13 févr. 2023 à 11:46, Joost Jager <joost.jager at gmail.com> a écrit :
> Hi,
>
> For a long time I've held the expectation that eventually payers on the
> lightning network will become very strict about node performance. That they
> will require a routing node to operate flawlessly or else apply a hefty
> penalty such as completely avoiding the node for an extended period of time
> - multiple weeks. The consequence of this is that routing nodes would need
> to manage their liquidity meticulously because every failure potentially
> has a large impact on future routing revenue.
>
> I think movement in this direction is important to guarantee
> competitiveness with centralised payment systems and their (at least
> theoretical) ability to process a payment in the blink of an eye. A
> lightning wallet trying multiple paths to find one that works doesn't help
> with this.
>
> A common argument against strict penalisation is that it would lead to
> less efficient use of capital. Routing nodes would need to maintain pools
> of liquidity to guarantee successes all the time. My opinion on this is
> that lightning is already enormously capital efficient at scale and that it
> is worth sacrificing a slight part of that efficiency to also achieve the
> lowest possible latency.
>
> This brings me to the actual subject of this post. Assuming strict
> penalisation is good, it may still not be ideal to flip the switch from one
> day to the other. Routing nodes may not offer the required level of service
> yet, causing senders to end up with no nodes to choose from.
>
> One option is to gradually increase the strength of the penalties, so that
> routing nodes are given time to adapt to the new standards. This does
> require everyone to move along and leaves no space for cheap routing nodes
> with less leeway in terms of liquidity.
>
> Therefore I am proposing another way to go about it: extend the
> `channel_update` field `channel_flags` with a new bit that the sender can
> use to signal `highly_available`.
>
> It's then up to payers to decide how to interpret this flag. One way could
> be to prefer `highly_available` channels during pathfinding. But if the
> routing node then returns a failure, a much stronger than normal penalty
> will be applied. For routing nodes this creates an opportunity to attract
> more traffic by marking some channels as `highly_available`, but it also
> comes with the responsibility to deliver.
>
> Without shadow channels, it is impossible to guarantee liquidity up to the
> channel capacity. It might make sense for senders to only assume high
> availability for amounts up to `htlc_maximum_msat`.
>
> A variation on this scheme that requires no extension of `channel_update`
> is to signal availability implicitly through routing fees. So the more
> expensive a channel is, the stronger the penalty that is applied on failure
> will be. It seems less ideal though, because it could disincentivize cheap
> but reliable channels on high traffic links.
>
> The effort required to implement some form of a `highly_available` flag
> seem limited and it may help to get payment success rates up. Interested to
> hear your thoughts.
>
> Joost
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230214/893867dd/attachment-0001.html>;
Antoine Riard [ARCHIVE] /
npub1vj…4x8dd
2023-06-09 13:08:08
in reply to nevent1q…z934
Author Public Key
npub1vjzmc45k8dgujppapp2ue20h3l9apnsntgv4c0ukncvv549q64gsz4x8ddPublished at
2023-06-09 13:08:08Event JSON
{
"id": "16aada6b6796c3d9b92291d52481ebd3d96f23ec1c17c6ff28e13ff4b0a35da3",
"pubkey": "6485bc56963b51c9043d0855cca9f78fcbd0ce135a195c3f969e18ca54a0d551",
"created_at": 1686316088,
"kind": 1,
"tags": [
[
"e",
"fc8a894a4e8d57d6b3b2f8ffd93a7c6696f3dc731595d0f0386959afdd2ef342",
"",
"root"
],
[
"e",
"d20ac0c4aefaee863d522440c267608e8d9e2ab173a8bd632d539053fe6c74f9",
"",
"reply"
],
[
"p",
"6485bc56963b51c9043d0855cca9f78fcbd0ce135a195c3f969e18ca54a0d551"
]
],
"content": "📅 Original date posted:2023-02-14\n📝 Original message:\nHi Joost,\n\n\u003e For a long time I've held the expectation that eventually payers on the\nlightning network will become very strict about node performance. That they\nwill \u003e require a routing node to operate flawlessly or else apply a hefty\npenalty such as completely avoiding the node for an extended period of time\n- multiple \u003e weeks. The consequence of this is that routing nodes would\nneed to manage their liquidity meticulously because every failure\npotentially has a large\n\u003e impact on future routing revenue.\n\nI think the performance question depends on the type of payment flows\nconsidered. If you're an\nend-user sending a payment to your local Starbucks for coffee, here fast\npayment sounds the end-goal.\nIf you're doing remittance payment, cheap fees might be favored, and in\nfunction of those flows you're\nprobably not going to select the same \"performant\" routing nodes. I think\nadding latency as a criteria for\npathfinding construction has already been mentioned in the past for LDK [0].\n\n\u003e I think movement in this direction is important to guarantee\ncompetitiveness with centralised payment systems and their (at least\ntheoretical) ability to\n\u003e process a payment in the blink of an eye. A lightning wallet trying\nmultiple paths to find one that works doesn't help with this.\n\nOr there is the direction to build forward-error-correction code on top of\nMPP, like in traditional\nnetworking [1]. The rough idea, you send more payment shards than the\nrequested sum, and then\nyou reveal the payment secrets to the receiver after an onion interactivity\nround to finalize payment.\n\n\u003e A common argument against strict penalisation is that it would lead to\nless efficient use of capital. Routing nodes would need to maintain pools of\n\u003e liquidity to guarantee successes all the time. My opinion on this is that\nlightning is already enormously capital efficient at scale and that it is\nworth\n\u003e sacrificing a slight part of that efficiency to also achieve the lowest\npossible latency.\n\nAt the end of the day, we add more signal channels between HTLC senders and\nthe routing\nnodes offering capital liquidity, if the signal mechanisms are efficient, I\nthink they should lead\nto better allocation of the capital. So yes, I think more liquidity might\nbe used by routing nodes\nto serve finely tailored HTLC requests by senders, however this liquidity\nshould be rewarded\nby higher routing fees.\n\n\u003e This brings me to the actual subject of this post. Assuming strict\npenalisation is good, it may still not be ideal to flip the switch from one\nday to the other. \u003e Routing nodes may not offer the required level of\nservice yet, causing senders to end up with no nodes to choose from.\n\n\u003e One option is to gradually increase the strength of the penalties, so\nthat routing nodes are given time to adapt to the new standards. This does\nrequire \u003e everyone to move along and leaves no space for cheap routing\nnodes with less leeway in terms of liquidity.\n\nI think if we have lessons to learn on policy rules design and deployment\non the base-layer\n(the full-rbf saga), it's to be careful in the initial set of rules, and\nhow we ensure smooth\nupgradeability, from one version to another. Otherwise the re-deployment\ncost towards\nthe new version might incentive the old routing node to stay on the\nnon-optimal versions,\nand as we have historical buckets in routing algorithms, or preference for\nolder channels,\nthis might lead the end-user to pay higher fees, than they could access to.\n\n\u003e Therefore I am proposing another way to go about it: extend the\n`channel_update` field `channel_flags` with a new bit that the sender can\nuse to signal \u003e `highly_available`.\n\n\u003e It's then up to payers to decide how to interpret this flag. One way\ncould be to prefer `highly_available` channels during pathfinding. But if\nthe routing\n\u003e node then returns a failure, a much stronger than normal penalty will be\napplied. For routing nodes this creates an opportunity to attract more\ntraffic by \u003e marking some channels as `highly_available`, but it also comes\nwith the responsibility to deliver.\n\nThis is where the open question lies to me - \"highly available\" can be\ndefined with multiple\nsenses, like fault-tolerance, latency processing, equilibrated liquidity.\nAnd a routing node might\nnot be able to optimize its architecture for the same end-goal (e.g more\nwatchtower on remote\nhost probably increases the latency processing).\n\n\u003e Without shadow channels, it is impossible to guarantee liquidity up to\nthe channel capacity. It might make sense for senders to only assume high\n\u003e availability for amounts up to `htlc_maximum_msat`.\n\nAs a note, I think \"senders assumption\" should be well-documented,\notherwise there will be\nperformance discrepancies between node implementations or even versions.\nE.g, an upgraded\nsender penalizing a node for the lack of shadow/parallel channels\nfulfilling HTLC amounts up to\n`htlc_maximum_msat`.\n\n\u003e A variation on this scheme that requires no extension of `channel_update`\nis to signal availability implicitly through routing fees. So the more\nexpensive \u003e a channel is, the stronger the penalty that is applied on\nfailure will be. It seems less ideal though, because it\ncould disincentivize cheap but reliable\n\u003e channels on high traffic links.\n\n\u003e The effort required to implement some form of a `highly_available` flag\nseem limited and it may help to get payment success rates up. Interested to\n\u003e hear your thoughts.\n\nI think signal availability should be explicit rather than implicit. Even\nif it's coming with more\ngossip bandwidth data consumed. I would say for bandwidth performance\nmanagement, relying\non new gossip messages, where they can be filtered in function of the level\nof services required\nis interesting.\n\nBest,\nAntoine\n\n[0] https://github.com/lightningdevkit/rust-lightning/issues/1647\n[1] https://www.rfc-editor.org/rfc/rfc6363.html\n\n\n\n\nLe lun. 13 févr. 2023 à 11:46, Joost Jager \u003cjoost.jager at gmail.com\u003e a écrit :\n\n\u003e Hi,\n\u003e\n\u003e For a long time I've held the expectation that eventually payers on the\n\u003e lightning network will become very strict about node performance. That they\n\u003e will require a routing node to operate flawlessly or else apply a hefty\n\u003e penalty such as completely avoiding the node for an extended period of time\n\u003e - multiple weeks. The consequence of this is that routing nodes would need\n\u003e to manage their liquidity meticulously because every failure potentially\n\u003e has a large impact on future routing revenue.\n\u003e\n\u003e I think movement in this direction is important to guarantee\n\u003e competitiveness with centralised payment systems and their (at least\n\u003e theoretical) ability to process a payment in the blink of an eye. A\n\u003e lightning wallet trying multiple paths to find one that works doesn't help\n\u003e with this.\n\u003e\n\u003e A common argument against strict penalisation is that it would lead to\n\u003e less efficient use of capital. Routing nodes would need to maintain pools\n\u003e of liquidity to guarantee successes all the time. My opinion on this is\n\u003e that lightning is already enormously capital efficient at scale and that it\n\u003e is worth sacrificing a slight part of that efficiency to also achieve the\n\u003e lowest possible latency.\n\u003e\n\u003e This brings me to the actual subject of this post. Assuming strict\n\u003e penalisation is good, it may still not be ideal to flip the switch from one\n\u003e day to the other. Routing nodes may not offer the required level of service\n\u003e yet, causing senders to end up with no nodes to choose from.\n\u003e\n\u003e One option is to gradually increase the strength of the penalties, so that\n\u003e routing nodes are given time to adapt to the new standards. This does\n\u003e require everyone to move along and leaves no space for cheap routing nodes\n\u003e with less leeway in terms of liquidity.\n\u003e\n\u003e Therefore I am proposing another way to go about it: extend the\n\u003e `channel_update` field `channel_flags` with a new bit that the sender can\n\u003e use to signal `highly_available`.\n\u003e\n\u003e It's then up to payers to decide how to interpret this flag. One way could\n\u003e be to prefer `highly_available` channels during pathfinding. But if the\n\u003e routing node then returns a failure, a much stronger than normal penalty\n\u003e will be applied. For routing nodes this creates an opportunity to attract\n\u003e more traffic by marking some channels as `highly_available`, but it also\n\u003e comes with the responsibility to deliver.\n\u003e\n\u003e Without shadow channels, it is impossible to guarantee liquidity up to the\n\u003e channel capacity. It might make sense for senders to only assume high\n\u003e availability for amounts up to `htlc_maximum_msat`.\n\u003e\n\u003e A variation on this scheme that requires no extension of `channel_update`\n\u003e is to signal availability implicitly through routing fees. So the more\n\u003e expensive a channel is, the stronger the penalty that is applied on failure\n\u003e will be. It seems less ideal though, because it could disincentivize cheap\n\u003e but reliable channels on high traffic links.\n\u003e\n\u003e The effort required to implement some form of a `highly_available` flag\n\u003e seem limited and it may help to get payment success rates up. Interested to\n\u003e hear your thoughts.\n\u003e\n\u003e Joost\n\u003e _______________________________________________\n\u003e Lightning-dev mailing list\n\u003e Lightning-dev at lists.linuxfoundation.org\n\u003e https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev\n\u003e\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230214/893867dd/attachment-0001.html\u003e",
"sig": "b9a287c8673a7119a2334a118bce856ce158661805b515d59fe2179f25158426de06d4ecd4247af44b6718fd5c58eba4a4f8388ffa632d1de7f1f2e007cb2332"
}