Kulpreet Singh [ARCHIVE] / is on Nostr

⚠️ This account is not associated with a real person. It's an archival account for mailing list messages.
🗓️ Active from 2018-06-12 to 2019-04-04
💬 Posted a total of 6 messages
📚 Participated in the following categories: bitcoin-dev, Lightning-dev
🔇 Please do not DM this user as it's used for archive purposes only.

Last Notes

2023-06-09 12:54:12

- reply

📅 Original date posted:2019-02-11
📝 Original message:
Hi Laolu,
Using the SURB approach to ACK payments definitely seems like a
sensible approach for all the reasons you listed.
I think this might be a good opportunity to think of an implicit ACK
mechanism too. It might not be useful in the immediate future, but
could reduce some traffic in the long run, especially for streaming
use cases.
I am assuming that streaming payments will encode payment sequence
numbers in the EOB or some place else.
The simple idea being: An ACK for paymentN implicitly ACKs payment1,
payment2 and so on up to paymentN-1 and including paymentN.
Implicit ACKs will help build streaming apps without increased traffic
carrying ACK packets for each sub cent payment.
With implicit ACKs available at LN layer, application libraries and
frameworks can develop streaming payment protocols that play around
with ACK window size etc.
For one off payments, the streaming sequence number is not set and
implicit ACKing is not triggered.
Just a suggestion, knowing quite well I am not aware of all the
complications involved.
Kulpreet
Kulpreet
On Fri, 8 Feb 2019 at 02:20, Olaoluwa Osuntokun <laolu32 at gmail.com> wrote:
>
> Hi y'all,
>
> Recently we've started to do more design work related to the Sphinx packet
> (EOB format, rendezvous protocol). This prompted me to re-visit the original
> Sphinx paper to refresh my memory w.r.t some of the finer details of the
> protocol. While I was re-reading the paper, I realized that we may be able
> to use use SURBs (single-use-reply-blocks) to implement a "payment ACK" for
> each sent HTLC.
>
> (it's worth mentioning that switching to HORNET down the line would solve
> this problem as well since the receiver already gets a multi-use backwards
> route that they can use to send information back to the receiver)
>
> Right now HTLC routing is mainly a game of "send and hope it arrives", as
> you have no clear indication of the _arrival_ of an HTLC at the destination.
> Instead, you only receive a protocol level message if the HTLC failed for
> w/e reason, or if it was successfully redeemed. As part of BOLT 1.1, it was
> agreed upon that we should implement some sort of "payment ACK" feature. A
> payment ACK scheme is strongly desired as it:
>
> * Allows the sender to actually know when a payment has reached the
> receiver which is useful for many higher level protocols. Atm, the
> sender is unable to distinguish an HTLC being "black holed" from one
> that's actually reached the sender, and they're just holding on to it.
> * AMP implementations would be aided by being able to receive feedback on
> successfully routed splits. If we're able to have the receiver ACK each
> partial payment, then implementations can more aggressively split
> payments as they're able to gain assurance that the first 2 BTC of 5
> total have actually reached the sender, and weren't black holed.
> * Enforcing and relying on ACKs may also thwart silly games receivers
> might play, claiming that the HTLC "didn't actually arrive".
>
> Some also call this feature a "soft error" as a possible implementation
> might to re-use the existing onion error protocol we've deployed today. For
> reference, in order to send back errors back long the route in a way that
> doesn't reveal the sender of the HTLC to the receiver (or any of the
> intermediate nodes) we re-use the shared secret each hop has derived, and
> onion wrap a MAC'd error to the sender. Each hop can't actually check that
> they've received a well formed error, but the sender is able to attribute an
> error to a node in the route based on which shared secret they're able to
> check the MAC with.
>
> The original Sphinx packet format has a way for the receiver to send a
> message back to the sender. This was originally envisioned to allow the
> receiver to send a replay email/message back to the sender without knowing
> who they were, and also in a manner that was bit-wise indistinguishable from
> a regular forwarded packet. This is called a SURB or "single use reply
> block". A SURB is composed of: a pre-crafted sphinx packet for the
> "backwards route" (which can be distinct from the forwards route), the first
> hop of the backwards route, and finally a symmetric key to use when
> encrypting the reply.
>
> When we more or less settled on using Sphinx, we started to remove things
> that we didn't have a clear use for at the time. Two things that were
> removed were the original end-to-end payload, and also the SURB. Removing
> the payload made the packet size smaller, and it didn't seem realistic to
> give _each_ hop a SURB to send reply back.
>
> In order to implement payment ACKs, we can have the sender craft a SURB (for
> the ACK), and mark the receipt of the SURB as the payment ACK itself.
> Creating and processing a SURB is identical to the regular HTLC packets we
> use today. As a result, the code impact to the code sphinx packet logic is
> minimal. We'd then also re-introduce the e2e payload so we can carry the
> SURB in the forward direction (HLTC add). The backwards packet would also
> have a payload of random bytes with the same size as a regular packet to
> make them look identical on the wire.
>
> This payload can further be put to use in order to implement streaming or
> subscription payments in a way. Since we must add a payload for in order to
> send/reply look the same, we can also piggy back some useful additional
> data. Each time a payment is sent, the receiver can use the extra payload to
> stack on details such as:
> * A new invoice to pay for the metered service being paid for.
> * An invoice along with a deadline for when this must be paid, lest the
> subscription service expire.
> * Details of lightning-native API
> * etc, etc
>
> IMO, this approach is better than a potential client-server payment
> negotiation protocol as it doesn't require any additional servers along side
> the node, also maintains sender anonymity, and doesn't rely on any sort of
> PKI.
>
> From the prospective of packet-analysis, errors today are identifiable due
> to the packet size (though we do pad them out to avoid being able to
> distinguish some errors from others on the wide). SURBs on the other hand,
> have the same profile as regular HTLC adds since they use the same Sphinx
> packet format. Unlike the wrapped onion errors, intermediate nodes are also
> able to validate the integrity of the payment ACK as they'll check the per-hop
> MAC as normal. Additionally, the replies naturally create cover traffic as
> they look like regular payments.
>
> One down side is that this would essentially double the size of HTLC
> messages on the network today, as they need to also carry SURB. Most
> candidates for possible rendezvous schemes to deploy would also increase the
> packet size if up to 20 hops it be allowed in both directions. We also
> (from my PoV), don't really have a feel on how much of an issue the 1.2KB
> HTLC packet size is today in the network.
>
> By re-introducing SURBs, we have an opportunity to cleanly solve the payment
> ACK issue, re-use a component of our favorite thing in Lightning, enable
> protocol-level streaming/subscription payments and also make replies
> indistinguishable-ish from regular payments.
>
> Thoughts?
>
> -- Laolu
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

2023-06-09 12:51:28

- reply

📅 Original date posted:2018-09-13
📝 Original message:
Hi Elias,
Thanks for the pointer to your excellent paper. I really enjoyed
reading the idea of using locked flows to allow for a distributed
execution for finding valid flows, which will enable discovering
routes for payments. Pretty neat idea indeed.
I just have few questions. My understanding might be incorrect and
therefore I ask.
1. The distributed execution as proposed in the Goldberg paper
requires a number of nodes to communicate with each other, essentially
nodes are talking to their neighbours, but I imagine this results in a
waterfall eventually as nodes talk their neighbours and so on. How do
you see this working practically? Do you envision periodic route
calculation phases, where all routing nodes collaborate even x seconds
to help find routes for all the payments in that phase? This might
result in the "instant payment" property of LN suffering a bit. I hope
I understood the Goldberg approach and my question isn't completely
off the hook.
2. If we leave the distributed execution aside for a moment and try
the "feasible flow" approach from your paper and compare that to the
current implementation in LN for route calculation, it might make for
an interesting result. So the question is, do you have an
implementation of the push-relable algorithm in Go that could be tried
out in LN? Or have you see one as part of another lib?
About my work to observe LN metrics, I was hoping to see if there were
any metrics from the network flow literature that might be relevant to
LN. Do you think any specific one might be interesting to track?
Thanks again for sharing your paper
Kulpreet
On Wed, 29 Aug 2018 at 11:45, Elias Rohrer <elias.rohrer at tu-berlin.de> wrote:
>
> Hello Kulpreet,
>
> thank you a lot for your work, your post highlights some really interesting results.
>
> I'm looking forward to your future work measuring the network's structure and decentralization. Moreover, I find thinking about the LN as a flow network to be a real interesting perspective. In fact, as an initial entry towards this direction, we did some work on a multi-path routing algorithm based on the push-relabel algorithm (cf. 1). However, I think there is much room for improvement and other flow algorithms could turn out to be promising candidates for routing in the LN.
>
> As far as I understood, you would be using flow algorithms for a more in-depth analysis of the networks graph structure?
>
> Kind Regards,
>
> Elias
>
> On 29 Aug 2018, at 8:40, Kulpreet Singh wrote:
>
> Hi René,
>
> Thanks for sharing the links to the issues and the excellent work you are doing.
>
> I see that you are quite interested in helping autopilot create a
> graph such that is provides certain characteristics. It is quite a
> challenging task, and I admire your courage to take it on. I saw your
> lib-autopilot too, though I didn't take a closer look at the code yet.
>
> I am focussing on a much simpler task of figuring out which metrics
> the community will find useful and then trying to determine which
> algorithms we could quickly run to get some initial results while we
> try to develop more pertinent analysis algorithms.
>
> I focussed on betweeness centrality and articulation points as
> personally I was most curious about those. Next I want to try and
> figure out which max-flow algorithm might suit LN the best. Have you
> looked at this? I might have missed something you might have already
> pointed to.
>
> I noticed your concern about tracking articulation points. I agree,
> that once central point dominance goes down the dependence on some
> important articulation points will go down too. But as I noticed in my
> results, some nodes are high in the list of articulation points sorted
> by the number of atleast 5 node biconnected components they connect
> to. But their centrality is not that high. For example,
> mainnet.lightningconductor.net is in the list of top articulation
> points but does not make it to the list of top 20 nodes by centrality.
>
> I am still curious about articulation points and want to keep tracking
> it, who knows we might learn something interesting by tracking the
> information.
>
> I am curious how are you running your graph evaluations. Are you using
> python binding to BGL or python networkx?
>
> I imagine we got slightly different results also because we used data
> from different times. I intend to publish the json output I got from
> LND when I get around to plotting my results on a chart so we can
> verify I am not making an error somewhere.
>
> Thanks and regards
> Kulpreet
>
>
> On Tue, 28 Aug 2018 at 00:31, René Pickhardt <r.pickhardt at googlemail.com> wrote:
>
> Hey Kulpreet,
>
> thanks for this nice overview article! I have just today implemented a first draft for the c-lightning autopilot [0]. I have implemented 4 heuristics to select nodes to which one could connect. On of those [1] samples from the nodes that contribute to the high diameter. This heuristic was included not to increase the utility of the node that is running the autopilot but to improve the network properties. I believe that this heuristic should also reduce the articulation points and biconnected components that you mention in your article. As endpoints of longest shortest paths will most likely be in two different biconnectivity components (at least if those exist and have a certain size).
>
> Regarding the centrality. I also calculated the betweeness centrality and have similar results [2] to yours. I guess the difference will be due to the fact that we don't work on the exact same snapshot. My autopilot implementation also connects to a few rather central nodes. I doubt this is useful for the network but I guess it is good for the node running the autopilot since it gains access to many nodes. ( Actually I think - but don't know - that in combination with [1] it even helps the network).
>
> Regarding your 200 Articulation points I would guess that many of those are just nodes that only have one channel with the node that acts as an articulation point. I guess this is not something that we would need to take care of so much since it is also in the responsibility of those nodes to have more than one channel. for larger biconnectivity components the problem would probably be resolved with the above mentioned heuristic. Therefor I believe looking at the articulation points should not be our main focus.
>
> Something that (regarding the autopilot) I am currently missing in your article is how much funds should be allocated for the suggested channels. I am currently experimenting with a probability density function that is proportional to the average capacity of each node in the candidate set. I smooth this with a uniform distribution. However simulations at this point are quite expensive (if done primitively since the centralities have to be recomputed) I guess this would be a nice future work. I will probably tomorrow publish the rest of the code for the lib-autopilot that uses this heuristic for channel balances since I am currently still working on it.
>
> If you consider working more on the autopilot but also on research related to this I would also suggest the following resources [3],[4] and [5]
>
> [0] https://github.com/ElementsProject/lightning/pull/1888
> [1] https://github.com/renepickhardt/lightning/blob/8c91f57490b51f772513a274d679d3ab62e7201a/contrib/lib-autopilot.py#L205
> [2] https://twitter.com/renepickhardt/status/1034066602273193985
> [3] https://github.com/lightningnetwork/lnd/issues/677
> [4] https://github.com/renepickhardt/Automatically-Generating-a-Robust-Topology-for-the-Lightning-Network-on-top-of-Bitcoin
> [5] https://www.rene-pickhardt.de/improve-the-autopilot-of-bitcoins-lightning-network-summary-of-the-bar-camp-session-at-the-2nd-lightninghackday-in-berlin/
>
> best regards Rene
>
>
> On Mon, Aug 27, 2018 at 11:59 PM Kulpreet Singh <zapfmann at gmail.com> wrote:
>
> Hi all,
>
> I have been thinking about how we could measure the centrality of
> various nodes in the LN graph and the dependence on some nodes to
> route payments and to prevent network partitions. I think measuring
> and tracking the changes in key metrics could help the community
> decide on which nodes to open new channels with.
>
> I measured the centrality of nodes and the central point dominance as
> defined in the seminal paper by Lindon C. Freeman, "A Set of Measures
> of Centrality Based on Betweenness", Sociometry 40, pp. 35-41, 1977.
>
> I also measured the number of articulation points in the network as
> per Robert E. Tarjan, "Depth first search and linear graph algorithms"
> SIAM Journal on Computing, 1(2):146-160, 1972.
>
> I want to add, that this is just a start, I understand that we should
> probably look at treating LN as a directed graph and that we should do
> some work in trying to do some analysis based on treating LN as a flow
> network.
>
> However, I am eager to share my early results and would welcome any
> feedback or suggestions on the way forward.
>
> I wrote a medium post describing the approach and show my results
> there. I also elaborate on the choice of the two metrics and what they
> mean for LN. The post is available here:
> https://medium.com/@jungly/measuring-node-centrality-in-lightning-network-8102a59999f0
>
> Looking forward to your suggestions and feedback.
>
> Thanks
> Kulpreet
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
> --
> https://www.rene-pickhardt.de
>
> Skype: rene.pickhardt
>
> mobile: +49 (0)176 5762 3618
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

2023-06-09 12:51:24

- reply

📅 Original date posted:2018-08-29
📝 Original message:
Hi René,
Thanks for sharing the links to the issues and the excellent work you are doing.
I see that you are quite interested in helping autopilot create a
graph such that is provides certain characteristics. It is quite a
challenging task, and I admire your courage to take it on. I saw your
lib-autopilot too, though I didn't take a closer look at the code yet.
I am focussing on a much simpler task of figuring out which metrics
the community will find useful and then trying to determine which
algorithms we could quickly run to get some initial results while we
try to develop more pertinent analysis algorithms.
I focussed on betweeness centrality and articulation points as
personally I was most curious about those. Next I want to try and
figure out which max-flow algorithm might suit LN the best. Have you
looked at this? I might have missed something you might have already
pointed to.
I noticed your concern about tracking articulation points. I agree,
that once central point dominance goes down the dependence on some
important articulation points will go down too. But as I noticed in my
results, some nodes are high in the list of articulation points sorted
by the number of atleast 5 node biconnected components they connect
to. But their centrality is not that high. For example,
mainnet.lightningconductor.net is in the list of top articulation
points but does not make it to the list of top 20 nodes by centrality.
I am still curious about articulation points and want to keep tracking
it, who knows we might learn something interesting by tracking the
information.
I am curious how are you running your graph evaluations. Are you using
python binding to BGL or python networkx?
I imagine we got slightly different results also because we used data
from different times. I intend to publish the json output I got from
LND when I get around to plotting my results on a chart so we can
verify I am not making an error somewhere.
Thanks and regards
Kulpreet
On Tue, 28 Aug 2018 at 00:31, René Pickhardt <r.pickhardt at googlemail.com> wrote:
>
> Hey Kulpreet,
>
> thanks for this nice overview article! I have just today implemented a first draft for the c-lightning autopilot [0]. I have implemented 4 heuristics to select nodes to which one could connect. On of those [1] samples from the nodes that contribute to the high diameter. This heuristic was included not to increase the utility of the node that is running the autopilot but to improve the network properties. I believe that this heuristic should also reduce the articulation points and biconnected components that you mention in your article. As endpoints of longest shortest paths will most likely be in two different biconnectivity components (at least if those exist and have a certain size).
>
> Regarding the centrality. I also calculated the betweeness centrality and have similar results [2] to yours. I guess the difference will be due to the fact that we don't work on the exact same snapshot. My autopilot implementation also connects to a few rather central nodes. I doubt this is useful for the network but I guess it is good for the node running the autopilot since it gains access to many nodes. ( Actually I think - but don't know - that in combination with [1] it even helps the network).
>
> Regarding your 200 Articulation points I would guess that many of those are just nodes that only have one channel with the node that acts as an articulation point. I guess this is not something that we would need to take care of so much since it is also in the responsibility of those nodes to have more than one channel. for larger biconnectivity components the problem would probably be resolved with the above mentioned heuristic. Therefor I believe looking at the articulation points should not be our main focus.
>
> Something that (regarding the autopilot) I am currently missing in your article is how much funds should be allocated for the suggested channels. I am currently experimenting with a probability density function that is proportional to the average capacity of each node in the candidate set. I smooth this with a uniform distribution. However simulations at this point are quite expensive (if done primitively since the centralities have to be recomputed) I guess this would be a nice future work. I will probably tomorrow publish the rest of the code for the lib-autopilot that uses this heuristic for channel balances since I am currently still working on it.
>
> If you consider working more on the autopilot but also on research related to this I would also suggest the following resources [3],[4] and [5]
>
> [0] https://github.com/ElementsProject/lightning/pull/1888
> [1] https://github.com/renepickhardt/lightning/blob/8c91f57490b51f772513a274d679d3ab62e7201a/contrib/lib-autopilot.py#L205
> [2] https://twitter.com/renepickhardt/status/1034066602273193985
> [3] https://github.com/lightningnetwork/lnd/issues/677
> [4] https://github.com/renepickhardt/Automatically-Generating-a-Robust-Topology-for-the-Lightning-Network-on-top-of-Bitcoin
> [5] https://www.rene-pickhardt.de/improve-the-autopilot-of-bitcoins-lightning-network-summary-of-the-bar-camp-session-at-the-2nd-lightninghackday-in-berlin/
>
> best regards Rene
>
>
> On Mon, Aug 27, 2018 at 11:59 PM Kulpreet Singh <zapfmann at gmail.com> wrote:
>>
>> Hi all,
>>
>> I have been thinking about how we could measure the centrality of
>> various nodes in the LN graph and the dependence on some nodes to
>> route payments and to prevent network partitions. I think measuring
>> and tracking the changes in key metrics could help the community
>> decide on which nodes to open new channels with.
>>
>> I measured the centrality of nodes and the central point dominance as
>> defined in the seminal paper by Lindon C. Freeman, "A Set of Measures
>> of Centrality Based on Betweenness", Sociometry 40, pp. 35-41, 1977.
>>
>> I also measured the number of articulation points in the network as
>> per Robert E. Tarjan, "Depth first search and linear graph algorithms"
>> SIAM Journal on Computing, 1(2):146-160, 1972.
>>
>> I want to add, that this is just a start, I understand that we should
>> probably look at treating LN as a directed graph and that we should do
>> some work in trying to do some analysis based on treating LN as a flow
>> network.
>>
>> However, I am eager to share my early results and would welcome any
>> feedback or suggestions on the way forward.
>>
>> I wrote a medium post describing the approach and show my results
>> there. I also elaborate on the choice of the two metrics and what they
>> mean for LN. The post is available here:
>> https://medium.com/@jungly/measuring-node-centrality-in-lightning-network-8102a59999f0
>>
>> Looking forward to your suggestions and feedback.
>>
>> Thanks
>> Kulpreet
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
>
>
> --
> https://www.rene-pickhardt.de
>
> Skype: rene.pickhardt
>
> mobile: +49 (0)176 5762 3618

2023-06-09 12:51:23

- reply

📅 Original date posted:2018-08-27
📝 Original message:
Hi all,
I have been thinking about how we could measure the centrality of
various nodes in the LN graph and the dependence on some nodes to
route payments and to prevent network partitions. I think measuring
and tracking the changes in key metrics could help the community
decide on which nodes to open new channels with.
I measured the centrality of nodes and the central point dominance as
defined in the seminal paper by Lindon C. Freeman, "A Set of Measures
of Centrality Based on Betweenness", Sociometry 40, pp. 35-41, 1977.
I also measured the number of articulation points in the network as
per Robert E. Tarjan, "Depth first search and linear graph algorithms"
SIAM Journal on Computing, 1(2):146-160, 1972.
I want to add, that this is just a start, I understand that we should
probably look at treating LN as a directed graph and that we should do
some work in trying to do some analysis based on treating LN as a flow
network.
However, I am eager to share my early results and would welcome any
feedback or suggestions on the way forward.
I wrote a medium post describing the approach and show my results
there. I also elaborate on the choice of the two metrics and what they
mean for LN. The post is available here:
https://medium.com/@jungly/measuring-node-centrality-in-lightning-network-8102a59999f0
Looking forward to your suggestions and feedback.
Thanks
Kulpreet

2023-06-07 18:17:29

- reply

📅 Original date posted:2019-04-04
📝 Original message:Hi Nicolas,
I have a small question about FastSync.
Would it make sense to validate all blocks once FastSync is complete
and BTCPayServer has started accepting payments?
I am aware this will require changes to bitcoind. So this is just an
academic question to figure if there are problems with such an
approach, especially for merchants accepting payments who want to get
started immediately and still want to stay on a Raspberry PI.
Phase 1: FastSync from trusted UTXO set and start accepting payments.
Phase 2: Validate the entire blockchain - this will take X days on
Raspberry PI - but at least in in the end you can fully trust your own
node. In this phase, you'd do IBD, but instead of writing to db, just
verify that the validated block matches the on the the db and move on.
It is a pity leveldb doesn't allow multiple processes to open the db.
If so, phase 2 could have been a different process altogether as well.
Regards
Kulpreet
On Wed, 3 Apr 2019 at 21:23, Nicolas Dorier via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> James,
>
> You might be interested by my work which is currently used in production, without any change to bitcoin core.
>
> I properly explain how to verify the utxoset independently.
>
> https://github.com/btcpayserver/btcpayserver-docker/blob/master/contrib/FastSync/README.md
>
> People are using it, since I get around 10 download a day.
> What can be done to help at Bitcoin Core level is actually very minimal.
>
> First, instead of asking signers of by UTXOSet to sign the utxoset hash from gettxoutsetinfo, I ask them to sign the hash of the tarball of my UTXO Set.
>
> The reason is that it is currently impossible to stop BitcoinD on a specific block then asking the serialized hash of the UTXO Set.
>
> So instead, a verifier download the tarball (300 blocks + utxoset at specific height), sync to the latest block, then compare the gettxoutsetinfo of the newly synched node with another trusted node. If it match, the verifier sign the tarball.
>
> I create a new utxoset snapshot every 6 months, so people have time to verify it and add their signatures. (Approximately once every bitcoin core release)
>
> The easiest thing that could be done at Bitcoin Core level does not require any code change, but a change in the release process.
>
> The new process would be to ask to the gitian signers to not only build the source themselves, but also verify a tarball following the procedure I explain in the link above.
>
> More complicated solution like signing the serialized utxoset itself, while possible, would require bothersome code changes.
>
> Nicolas,
>
> On Wed, Apr 3, 2019 at 9:25 AM <bitcoin-dev-request at lists.linuxfoundation.org> wrote:
>>
>> Send bitcoin-dev mailing list submissions to
>> bitcoin-dev at lists.linuxfoundation.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> or, via email, send a message with subject or body 'help' to
>> bitcoin-dev-request at lists.linuxfoundation.org
>>
>> You can reach the person managing the list at
>> bitcoin-dev-owner at lists.linuxfoundation.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of bitcoin-dev digest..."
>>
>>
>> Today's Topics:
>>
>> 1. BIP: Bitcoin Integrated Address Feature? (nathanw at tutanota.com)
>> 2. Re: BIP: Bitcoin Integrated Address Feature? (htimSxelA)
>> 3. assumeutxo and UTXO snapshots (James O'Beirne)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Tue, 2 Apr 2019 18:53:11 +0200 (CEST)
>> From: <nathanw at tutanota.com>
>> To: <bitcoin-dev at lists.linuxfoundation.org>
>> Subject: [bitcoin-dev] BIP: Bitcoin Integrated Address Feature?
>> Message-ID: <LbTxyE4--3-1 at tutanota.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> To whom it may concern,
>>
>> I believe a missing feature in Bitcoin is the ability to have an "integrated address", where the address resolves into a Bitcoin address, and also a transaction message or some other kind of identifier.
>>
>> By having this feature we could enhance the security of exchange cold-wallet systems, by allowing them to easily receive all payments to a single address from an infinite number of customers. We would also greatly simplify the process of setting up and managing exchange cold-wallet systems, because we would eliminate the "sweeping" step required to move multiple customer deposits from a hot address into a single cold address.
>>
>> Although it would be nice to have all customers deposit directly into cold addresses, this quickly becomes impractical when large amounts of customers begin to use exchange wallets as their personal web-wallet, frequently depositing and withdrawing without trading action. You end up needing to have a staff member moving funds away from cold deposit addresses as a full time job - if you wish to handle customer funds in a completely secure manner.
>>
>> Thus we see that most exchanges now use the hot-deposit system, where customers deposit into a hot address that is then automatically swept into a singular cold address, by a service which holds customers private keys online. You can observe this service at work simply by making a deposit to most major exchanges (including the largest exchange Binance), as you will see the funds quickly being "swept" to their cold wallet address in a manner which heavily suggests automation by a program which possesses private keys to the address you are sending funds to. This means there is always the danger of a sophisticated hacker being able to capture private keys to customer deposit addresses (as they are clearly being held online). An integrated address would allow all exchanges using this automated hot-deposit service to easily switch to a far more secure alternative of having all customers depositing directly into their singular cold wallet address.
>>
>> There are several other more minor advantages such a feature would have, including:
>> - Lower fees for exchanges (which could be passed onto customers), by reducing a transaction step out of the deposit-to-withdrawal flow.
>> - Less need for large rescans after loading huge amounts of customer addresses into client software.
>> - Exchanges can more easily provision deposit addresses to new customers in a secure manner, by simply generating a hex or other value, creating an integrated address from the cold wallet address, and then providing this to the customer.
>> - By providing a singular cold address for exchanges publicly, customers can more easily verify that no man-in-the-middle has given them an incorrect address to deposit to.
>> The integrated address could work by combining the Bitcoin address together with some kind of hex or other value, allowing users to choose the amount they wish to deposit themselves, but ensuring their deposits are uniquely trackable.
>>
>> I'm not sure if some kind of functionality already exists in BTC, as I haven't been able to find it. If not, can I submit a proposal to implement this? This feature would be a godsend to all exchange developers if it was widely accepted.
>>
>> Thanks for your time.
>> Regards,
>>
>> Nathan Worsley
>> CTO - LocalCoinSwap.Com
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190402/400c1e1b/attachment-0001.html>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Tue, 02 Apr 2019 20:01:34 +0000
>> From: htimSxelA <htimsxela at protonmail.com>
>> To: "nathanw at tutanota.com" <nathanw at tutanota.com>, Bitcoin Protocol
>> Discussion <bitcoin-dev at lists.linuxfoundation.org>
>> Subject: Re: [bitcoin-dev] BIP: Bitcoin Integrated Address Feature?
>> Message-ID:
>> <wtbAF1FAGePDAkY3xkqANuFJtAhEXvz0JeGWnc_OZcGEyFQb-1B590I3IbwtW2FBivur0yONbSQtxaWqiQTJeoDdadivtbGkWwJnLnnzQQE=@protonmail.com>
>>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hello,
>>
>> I see two immediate issues with this:
>> 1. Increased resource requirements per transaction
>> 2. Embedding identifying information into the blockchain is generally bad for privacy
>>
>> It may help your case to provide some technical details of how you'd like to see this implemented, but without overcoming the issues mentioned above I think this proposal will be a very tough sell.
>>
>> > ...this quickly becomes impractical when large amounts of customers begin to use exchange wallets as their personal web-wallet, frequently depositing and withdrawing without trading action. You end up needing to have a staff member moving funds away from cold deposit addresses as a full time job - if you wish to handle customer funds in a completely secure manner.
>>
>> I am not sure if I see how this issue is solved by your proposal. Assumedly, a human will still need to manually approve cold-wallet withdrawals in order to maintain security. So it seems to me that removing the 'hot-wallet' component of the backend would only amplify the need for human interaction.
>>
>> I assume you are familiar with hierarchical deterministic wallets? They can allow an exchange to assign/identify user deposits based on address derivation path. Keys for deposit addresses can be kept offline if wanted, and a proper implementation of an HD wallet system should also remove the need for rescans of user deposit addresses.
>>
>> There is also a functionality built into Bitcoin that allows a user to prove that they own the private keys to some address: signing an agreed upon message using the private key that controls that address. Unfortunately I don't think this is a workable solution for you, since the majority of modern wallet software does not include this feature-- but perhaps worth mentioning nonetheless.
>>
>> Best,
>> Alex
>>
>> ??????? Original Message ???????
>> On Tuesday, April 2, 2019 9:53 AM, Nathan Worsley via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>> > To whom it may concern,
>> >
>> > I believe a missing feature in Bitcoin is the ability to have an "integrated address", where the address resolves into a Bitcoin address, and also a transaction message or some other kind of identifier.
>> >
>> > By having this feature we could enhance the security of exchange cold-wallet systems, by allowing them to easily receive all payments to a single address from an infinite number of customers. We would also greatly simplify the process of setting up and managing exchange cold-wallet systems, because we would eliminate the "sweeping" step required to move multiple customer deposits from a hot address into a single cold address.
>> >
>> > Although it would be nice to have all customers deposit directly into cold addresses, this quickly becomes impractical when large amounts of customers begin to use exchange wallets as their personal web-wallet, frequently depositing and withdrawing without trading action. You end up needing to have a staff member moving funds away from cold deposit addresses as a full time job - if you wish to handle customer funds in a completely secure manner.
>> >
>> > Thus we see that most exchanges now use the hot-deposit system, where customers deposit into a hot address that is then automatically swept into a singular cold address, by a service which holds customers private keys online. You can observe this service at work simply by making a deposit to most major exchanges (including the largest exchange Binance), as you will see the funds quickly being "swept" to their cold wallet address in a manner which heavily suggests automation by a program which possesses private keys to the address you are sending funds to. This means there is always the danger of a sophisticated hacker being able to capture private keys to customer deposit addresses (as they are clearly being held online). An integrated address would allow all exchanges using this automated hot-deposit service to easily switch to a far more secure alternative of having all customers depositing directly into their singular cold wallet address.
>> >
>> > There are several other more minor advantages such a feature would have, including:
>> > - Lower fees for exchanges (which could be passed onto customers), by reducing a transaction step out of the deposit-to-withdrawal flow.
>> > - Less need for large rescans after loading huge amounts of customer addresses into client software.
>> > - Exchanges can more easily provision deposit addresses to new customers in a secure manner, by simply generating a hex or other value, creating an integrated address from the cold wallet address, and then providing this to the customer.
>> > - By providing a singular cold address for exchanges publicly, customers can more easily verify that no man-in-the-middle has given them an incorrect address to deposit to.
>> >
>> > The integrated address could work by combining the Bitcoin address together with some kind of hex or other value, allowing users to choose the amount they wish to deposit themselves, but ensuring their deposits are uniquely trackable.
>> >
>> > I'm not sure if some kind of functionality already exists in BTC, as I haven't been able to find it. If not, can I submit a proposal to implement this? This feature would be a godsend to all exchange developers if it was widely accepted.
>> >
>> > Thanks for your time.
>> >
>> > Regards,
>> >
>> > Nathan Worsley
>> > CTO - LocalCoinSwap.Com
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190402/ade34235/attachment-0001.html>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Tue, 2 Apr 2019 16:43:11 -0400
>> From: "James O'Beirne" <james.obeirne at gmail.com>
>> To: bitcoin-dev at lists.linuxfoundation.org
>> Subject: [bitcoin-dev] assumeutxo and UTXO snapshots
>> Message-ID:
>> <CAPfvXf+JS6ZhXUieWVxiaNa4uhhWwafCk3odMKy5F_yi=XwngA at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hi,
>>
>> I'd like to discuss assumeutxo, which is an appealing and simple
>> optimization in the spirit of assumevalid[0].
>>
>> # Motivation
>>
>> To start a fully validating bitcoin client from scratch, that client
>> currently
>> needs to perform an initial block download. To the surprise of no one, IBD
>> takes a linear amount time based on the length of the chain's history. For
>> clients running on modest hardware under limited bandwidth constraints,
>> say a mobile device, completing IBD takes a considerable amount of time
>> and thus poses serious usability challenges.
>>
>> As a result, having fully validating clients run on such hardware is rare
>> and
>> basically unrealistic. Clients with even moderate resource constraints
>> are encouraged to rely on the SPV trust model. Though we have promising
>> improvements to existing SPV modes pending deployment[1], it's worth
>> thinking about a mechanism that would allow such clients to use trust
>> models closer to full validation.
>>
>> The subject of this mail is a proposal for a complementary alternative to
>> SPV
>> modes, and which is in the spirit of an existing default, `assumevalid`. It
>> may
>> help modest clients transact under a security model that closely resembles
>> full validation within minutes instead of hours or days.
>>
>> # assumeutxo
>>
>> The basic idea is to allow nodes to initialize using a serialized version
>> of the
>> UTXO set rendered by another node at some predetermined height. The
>> initializing node syncs the headers chain from the network, then obtains and
>> loads one of these UTXO snapshots (i.e. a serialized version of the UTXO set
>> bundled with the block header indicating its "base" and some other
>> metadata).
>>
>> Based upon the snapshot, the node is able to quickly reconstruct its
>> chainstate,
>> and compares a hash of the resulting UTXO set to a preordained hash
>> hard-coded
>> in the software a la assumevalid. This all takes ~23 minutes, not
>> accounting for
>> download of the 3.2GB snapshot[2].
>>
>> The node then syncs to the network tip and afterwards begins a simultaneous
>> background validation (i.e., a conventional IBD) up to the base height of
>> the
>> snapshot in order to achieve full validation. Crucially, even while the
>> background validation is happening the node can validate incoming blocks and
>> transact with the benefit of the full (assumed-valid) UTXO set.
>>
>> Snapshots could be obtained from multiple separate peers in the same manner
>> as
>> block download, but I haven't put much thought into this. In concept it
>> doesn't
>> matter too much where the snapshots come from since their validity is
>> determined via content hash.
>>
>> # Security
>>
>> Obviously there are some security implications due consideration. While this
>> proposal is in the spirit of assumevalid, practical attacks may become
>> easier.
>> Under assumevalid, a user can be tricked into transacting under a false
>> history
>> if an attacker convinces them to start bitcoind with a malicious
>> `-assumevalid`
>> parameter, sybils their node, and then feeds them a bogus chain encompassing
>> all of the hard-coded checkpoints[3].
>>
>> The same attack is made easier in assumeutxo because, unlike in assumevalid,
>> the attacker need not construct a valid PoW chain to get the victim's node
>> into
>> a false state; they simply need to get the user to accept a bad
>> `-assumeutxo`
>> parameter and then supply them an easily made UTXO snapshot containing,
>> say, a
>> false coin assignment.
>>
>> For this reason, I recommend that if we were to implement assumeutxo, we not
>> allow its specification via commandline argument[4].
>>
>> Beyond this risk, I can't think of material differences in security
>> relative to
>> assumevalid, though I appeal to the list for help with this.
>>
>> # More fully validating clients
>>
>> A particularly exciting use-case for assumeutxo is the possibility of mobile
>> devices functioning as fully validating nodes with access to the complete
>> UTXO
>> set (as an alternative to SPV models). The total resource burden needed to
>> start a node
>> from scratch based on a snapshot is, at time of writing, a ~(3.2GB
>> + blocks_to_tip * 4MB) download and a few minutes of processing time, which
>> sounds
>> manageable for many mobile devices currently in use.
>>
>> A mobile user could initialize an assumed-valid bitcoin node within an hour,
>> transact immediately, and complete a pruned full validation of their
>> assumed-valid chain over the next few days, perhaps only doing the
>> background
>> IBD when their device has access to suitable high-bandwidth connections.
>>
>> If we end up implementing an accumulator-based UTXO scaling design[5][6]
>> down
>> the road, it's easy to imagine an analogous process that would allow very
>> fast
>> startup using an accumulator of a few kilobytes in lieu of a multi-GB
>> snapshot.
>>
>> ---
>>
>> I've created a related issue at our Github repository here:
>> https://github.com/bitcoin/bitcoin/issues/15605
>>
>> and have submitted a draft implementation of snapshot usage via RPC here:
>> https://github.com/bitcoin/bitcoin/pull/15606
>>
>> I'd like to discuss here whether this is a good fit for Bitcoin
>> conceptually. Concrete
>> plans for deployment steps should be discussed in the Github issue, and
>> after all
>> that my implementation may be reviewed as a sketch of the specific software
>> changes necessary.
>>
>> Regards,
>> James
>>
>>
>> [0]:
>> https://bitcoincore.org/en/2017/03/08/release-0.14.0/#assumed-valid-blocks
>> [1]: https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki
>> [2]: as tested at height 569895, on a 12 core Intel Xeon Silver 4116 CPU @
>> 2.10GHz
>> [3]:
>> https://github.com/bitcoin/bitcoin/blob/84d0fdc/src/chainparams.cpp#L145-L161
>> [4]: Marco Falke is due credit for this point
>> [5]: utreexo: https://www.youtube.com/watch?v=edRun-6ubCc
>> [6]: Boneh, Bunz, Fisch on accumulators: https://eprint.iacr.org/2018/1188
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190402/46b25dd8/attachment.html>
>>
>> ------------------------------
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>> End of bitcoin-dev Digest, Vol 47, Issue 6
>> ******************************************
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

2023-06-07 18:13:00

- reply

📅 Original date posted:2018-06-12
📝 Original message:Apologies for a noob question.
But if I understand correctly, lightning nodes need to check if a
counterparty is broadcasting an old channel state and in response broadcast
a penalty/justice transaction. Does that mean lightning nodes only need to
watch for transactions that come after the funding transaction? Is that the
only reason lightning needs to run bitcoind with txindex?
If that is the case, and a lightning node only needs to query transactions
broadcast after the funding transaction, then a pruned bitcoind instance
with txindex might be a bit handy.
Also from [1] it seems that indexing pruned nodes is not supported because
it doesn't make sense, not that it was infeasible. Now with the lightning
requirements, does an indexed pruned node start to make sense?
Once again, please forgive my naive understanding of some of the issues
involved and thanks for your patience.
Regards
Kulpreet
[1]
https://bitcoin.stackexchange.com/questions/52889/bitcoin-core-txindex-vs-default-mode-vs-pruned-mode-in-depth#52894
On Thu, 10 May 2018, 14:47 Patrick Shirkey via bitcoin-dev, <
bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> > On 3/17/18, someone posted on the Lightning-dev list, "Can I try
> > Lightning without running a fully-fledged bitcoin block chain? (Yubin
> > Ruan)." The inquirer was asking because he didn't have much space to
> > store the entire blockchain on his laptop.
> >
> > I replied:
> >
> > "Developers,
> >
> > On THIS note and slightly off-topic but relevant, why can't chunks of
> > blockchain peel off the backend periodically and be archived, say on
> > minimum of 150 computers across 7 continents?
> >
> > It seems crazy to continue adding on to an increasingly long chain to
> > infinity if the old chapters (i.e. more than, say, 2 years old) could be
> > stored in an evenly distributed manner across the planet. The same 150
> > computers would not need to store every chapter either, just the index
> > would need to be widely distributed in order to reconnect with a chapter
> > if needed. Then maybe it is no longer a limitation in the future for
> > people like Yubin. "
> >
> > It was suggested by a couple of lightning developers that I post this
> > idea on the bitcoin-dev list. So, here I post :).
> >
>
> You can already use the "prune" flag to get a snapshot of the blockchain
> but it is incompatible with "txindex" and "rescan" so maybe that is and
> issue for lightning nodes?
>
>
>
>
> --
> Patrick Shirkey
> Boost Hardware
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180612/242c770f/attachment.html>