Olaoluwa Osuntokun [ARCHIVE] on Nostr: 📅 Original date posted:2018-02-25 📝 Original message: > With that said, this ...

📅 Original date posted:2018-02-25
📝 Original message:
> With that said, this should instead be a distinct `chan_update_horizon`
> message (or w/e name). If a particular bit is set in the `init` message,
> then the next message *both* sides send *must* be `chan_update_horizon`.

Tweaking this a bit, if we make it: don't send *any* channel updates at all
unless the other side sends this message, then this allows both parties to
precisely control their initial load, and also if they even *want*
channel_update messages at all.

Purely routing nodes don't need any updates at all. In the case they wish to
send (assumed to be infrequent in this model), they'll get the latest update
after their first failure.

Similarly, leaf/edge nodes can opt to receive the latest updates if they
wish to minimize payment latency due to routing failures that are the result
of dated information.

IMO, the only case where a node would want the most up to date link policy
state is for optimization/analysis, or to minimize payment latency at the
cost of additional load.

--Laolu

On Fri, Feb 23, 2018 at 4:45 PM Olaoluwa Osuntokun <laolu32 at gmail.com>
wrote:

> Hi Rusty,
>
> > 1. query_short_channel_id
> > IMPLEMENTATION: trivial
>
> *thumbs up*
>
> > 2. query_channel_range/reply_channel_range
> > IMPLEMENTATION: requires channel index by block number, zlib
>
> For the sake of expediency of deployment, if we add a byte (or two) to
> denote the encoding/compression scheme, we can immediately roll out the
> vanilla (just list the ID's), then progressively roll out more
> context-specific optimized schemes.
>
> > 3. A gossip_timestamp field in `init`
> > This is a new field appended to `init`: the negotiation of this feature
> bit
> > overrides `initial_routing_sync`
>
> As I've brought up before, from my PoV, we can't append any additional
> fields to the innit message as it already contains *two* variable sized
> fields (and no fixed size fields). Aside from this, it seems that the
> `innit` message should be simply for exchange versioning information,
> which
> may govern exactly *which* messages are sent after it. Otherwise, by adding
> _additional_ fields to the `innit` message, we paint ourselves in a corner
> and can never remove it. Compared to using the `innit` message to set up
> the
> initial session context, where we can safely add other bits to nullify or
> remove certain expected messages.
>
> With that said, this should instead be a distinct `chan_update_horizon`
> message (or w/e name). If a particular bit is set in the `init` message,
> then the next message *both* sides send *must* be `chan_update_horizon`.
>
> Another advantage of making this a distinct message, is that either party
> can at any time update this horizon/filter to ensure that they only receive
> the *freshest* updates.Otherwise, one can image a very long lived
> connections (say weeks) and the remote party keeps sending me very dated
> updates (wasting bandwidth) when I only really want the *latest*.
>
> This can incorporate decker's idea about having a high+low timestamp. I
> think this is desirable as then for the initial sync case, the receiver can
> *precisely* control their "verification load" to ensure they only process a
> particular chunk at a time.
>
>
> Fabrice wrote:
> > We could add a `data` field which contains zipped ids like in
> > `reply_channel_range` so we can query several items with a single
> message ?
>
> I think this is an excellent idea! It would allow batched requests in
> response to a channel range message. I'm not so sure we need to jump
> *straight* to compressing everything however.
>
> > We could add an additional `encoding_type` field before `data` (or it
> > could be the first byte of `data`)
>
> Great minds think alike :-)
>
>
> If we're in rough agreement generally about this initial "kick can"
> approach, I'll start implementing some of this in a prototype branch for
> lnd. I'm very eager to solve the zombie churn, and initial burst that can
> be
> very hard on light clients.
>
> -- Laolu
>
>
> On Wed, Feb 21, 2018 at 10:03 AM Fabrice Drouin <fabrice.drouin at acinq.fr>
> wrote:
>
>> On 20 February 2018 at 02:08, Rusty Russell <rusty at rustcorp.com.au>
>> wrote:
>> > Hi all,
>> >
>> > This consumed much of our lightning dev interop call today! But
>> > I think we have a way forward, which is in three parts, gated by a new
>> > feature bitpair:
>>
>> We've built a prototype with a new feature bit `channel_range_queries`
>> and the following logic:
>> When you receive their init message and check their local features
>> - if they set `initial_routing_sync` and `channel_range_queries` then
>> do nothing (they will send you a `query_channel_range`)
>> - if they set `initial_routing_sync` and not `channel_range_queries`
>> then send your routing table (as before)
>> - if you support `channel_range_queries` then send a
>> `query_channel_range` message
>>
>> This way new and old nodes should be able to understand each other
>>
>> > 1. query_short_channel_id
>> > =========================
>> >
>> > 1. type: 260 (`query_short_channel_id`)
>> > 2. data:
>> > * [`32`:`chain_hash`]
>> > * [`8`:`short_channel_id`]
>>
>> We could add a `data` field which contains zipped ids like in
>> `reply_channel_range` so we can query several items with a single
>> message ?
>>
>> > 1. type: 262 (`reply_channel_range`)
>> > 2. data:
>> > * [`32`:`chain_hash`]
>> > * [`4`:`first_blocknum`]
>> > * [`4`:`number_of_blocks`]
>> > * [`2`:`len`]
>> > * [`len`:`data`]
>>
>> We could add an additional `encoding_type` field before `data` (or it
>> could be the first byte of `data`)
>>
>> > Appendix A: Encoding Sizes
>> > ==========================
>> >
>> > I tried various obvious compression schemes, in increasing complexity
>> > order (see source below, which takes stdin and spits out stdout):
>> >
>> > Raw = raw 8-byte stream of ordered channels.
>> > gzip -9: gzip -9 of raw.
>> > splitgz: all blocknums first, then all txnums, then all
>> outnums, then gzip -9
>> > delta: CVarInt encoding:
>> blocknum_delta,num,num*txnum_delta,num*outnum.
>> > deltagz: delta, with gzip -9
>> >
>> > Corpus 1: LN mainnet dump, 1830 channels.[1]
>> >
>> > Raw: 14640 bytes
>> > gzip -9: 6717 bytes
>> > splitgz: 6464 bytes
>> > delta: 6624 bytes
>> > deltagz: 4171 bytes
>> >
>> > Corpus 2: All P2SH outputs between blocks 508000-508999 incl, 790844
>> channels.[2]
>> >
>> > Raw: 6326752 bytes
>> > gzip -9: 1861710 bytes
>> > splitgz: 964332 bytes
>> > delta: 1655255 bytes
>> > deltagz: 595469 bytes
>> >
>> > [1] http://ozlabs.org/~rusty/short_channels-mainnet.xz
>> > [2] http://ozlabs.org/~rusty/short_channels-all-p2sh-508000-509000.xz
>> >
>>
>> Impressive!
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180225/3253059c/attachment-0001.html>;