Suhas Daftuar [ARCHIVE]

⚠️ This account is not associated with a real person. It's an archival account for mailing list messages.
🗓️ Active from 2015-07-30 to 2022-11-07
💬 Posted a total of 24 messages
📚 Participated in the following categories: bitcoin-dev, BIP Proposal
🔇 Please do not DM this user as it's used for archive purposes only.

Public Key

npub1ss2s89s938xfh3zzz0j2mhp25tlrlcrj0duljr0ne0t8v2r4cv0q5e0am4

Profile Code

nprofile1qqsgg9grjczcnnymc3pp8e9dms429l3lupe8k70epheuh4nk9p6ux8sdh40r3

Author Public Key

npub1ss2s89s938xfh3zzz0j2mhp25tlrlcrj0duljr0ne0t8v2r4cv0q5e0am4

Show more details

Last Notes

2023-06-07 23:16:44

- reply

📅 Original date posted:2022-11-07
📝 Original message:Hello bitcoin-dev,
The description in the OP is completely broken for the simple reason that
Bitcoin transactions can have multiple inputs, and so a single transaction
can conflict with multiple in-mempool transactions. The proposal would
limit the number of descendants of each in-mempool transaction to
MAX_REPLACEMENT_CANDIDATES (note that this is duplicative of the existing
Bitcoin Core descendant limits), but a transaction that has, say, 500
inputs might arrive and conflict with 500 unrelated in-mempool
transactions. This could result in 12,500 evictions -- far more than the
100 that was intended.
Also, note that those 500 transactions might instead have 24 ancestors each
(again, using the default chain limits in Bitcoin Core) -- this would
result in 12,000 transactions whose state would be updated as a result of
evicting those 500 transactions. Hopefully this gives some perspective on
why we have a limit.
On Mon, Nov 7, 2022 at 3:17 PM Peter Todd via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> tl;dr: We can remove the problem of Rule #5 pinning by ensuring that all
> transactions in the mempool are always replaceable.
>
>
> Currently Bitcoin Core implements rule #5 of BIP-125:
>
> The number of original transactions to be replaced and their descendant
> transactions which will be evicted from the mempool must not exceed a
> total
> of 100 transactions.
>
> As of Bitcoin Core v24.0rc3, this rule is implemented via the
> MAX_REPLACEMENT_CANDIDATES limit in GetEntriesForConflicts:
>
> // Rule #5: don't consider replacing more than
> MAX_REPLACEMENT_CANDIDATES
> // entries from the mempool. This potentially overestimates the number
> of actual
> // descendants (i.e. if multiple conflicts share a descendant, it will
> be counted multiple
> // times), but we just want to be conservative to avoid doing too much
> work.
> if (nConflictingCount > MAX_REPLACEMENT_CANDIDATES) {
> return strprintf("rejecting replacement %s; too many potential
> replacements (%d > %d)\n",
> txid.ToString(),
> nConflictingCount,
> MAX_REPLACEMENT_CANDIDATES);
> }
>
> There is no justification for this rule beyond avoiding "too much work";
> the
> exact number was picked at random when the BIP was written to provide an
> initial conservative limit, and is not justified by benchmarks or memory
> usage
> calculations. It may in fact be the case that this rule can be removed
> entirely
> as the overall mempool size limits naturally limit the maximum number of
> possible replacements.
>
>
> But for the sake of argument, let's suppose some kind of max replacement
> limit
> is required. Can we avoid rule #5 pinning? The answer is yes, we can, with
> the
> following invariant:
>
> No transaction may be accepted into the mempool that would cause
> another
> transaction to be unable to be replaced due to Rule #5.
>
> We'll call this the Replaceability Invariant. Implementing this rule is
> simple:
> for each transaction maintain an integer, nReplacementCandidates. When a
> non-conflicting transaction is considered for acceptance to the mempool,
> verify
> that nReplacementCandidates + 1 < MAX_REPLACEMENT_CANDIDATES for each
> unconfirmed parent. When a transaction is accepted, increment
> nReplacementCandidates by 1 for each unconfirmed parent.
>
> When a *conflicting* transaction is considered for acceptance, we do _not_
> need
> to verify anything: we've already guaranteed that every transaction in the
> mempool is replaceable! The only thing we may need to do is to decrement
> nReplacementCandidates by however many children we have replaced, for all
> unconfirmed parents.
>
> When a block is mined, note how the nReplacementCandidates of all
> unconfirmed
> transactions remains unchanged because a confirmed transaction can't spend
> an
> unconfirmed txout.
>
> The only special case to handle is a reorg that changes a transaction from
> confirmed to unconfirmed. Setting nReplacementCandidates to
> MAX_REPLACEMENT_CANDIDATES would be fine in that very rare case.
>
>
> Note that like the existing MAX_REPLACEMENT_CANDIDATES check, the
> Replaceability Invariant overestimates the number of transactions to be
> replaced in the event that multiple children are spent by the same
> transaction.
> That's ok: diamond tx graphs are even more unusual than unconfirmed
> children,
> and there's no reason to bend over backwards to accomodate them.
>
> Prior art: this proposed rule is similar in spirit to the package limits
> aready
> implemented in Bitcoin Core.
>
> --
> https://petertodd.org 'peter'[:-1]@petertodd.org
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20221107/200b5236/attachment.html>

2023-06-07 23:16:03

- reply

📅 Original date posted:2022-10-31
📝 Original message:AJ,
Thanks for the thoughtful post. I think your observations about how we view
mempool policy in the Bitcoin Core project, and how that seems to be
changing in the discussions around `-mempoolfullrbf`, are on-point and
provide a helpful baseline for considering future policy changes.
For a long time I viewed fullrbf as an eventuality and I considered myself
to be philosophically supportive of the idea. However, after giving this
issue some thought in the past few weeks, I am reversing my thinking on
this. Concretely, I will argue that we should continue to maintain a relay
policy where replacements are rejected for transactions that don't opt-in
to RBF (as described in BIP 125), and moreover, that we should remove the
`-mempoolfullrbf` flag from Bitcoin Core’s latest release candidate and not
plan to release software with that flag, unless (or until) circumstances
change on the network, which I'll discuss below.
This is, of course, a nuanced topic, and among the considerations is a
philosophy of how to think about the relay policy and configuration options
that we make available in Bitcoin Core (a consideration that is perhaps
unique to that project, but I think relevant for this mailing list).
I'll start with some technical issues regarding the benefits of enabling
fullrbf on the network. In the current BIP 125 regime, every time a
transaction is created, a choice is made whether to subject the transaction
to BIP 125’s RBF rules or not (based on the sequence values of the
inputs). So given that users can already opt-in to RBF, the benefit of a
“fullrbf” network policy would be if, somehow, RBF users were still denied
the benefits of RBF due to the existence of other transactions that don’t
opt-in.
Along those lines, Antoine Riard brought up[1] a DoS vector that is
available to someone who wants to interfere with multi-party funded
transactions, and suggested that fullrbf would eliminate the problem.
After exploring that question again in this thread (thanks to Greg Sanders
for clarifying this to me), I understand that the issue is around ensuring
that a multiparty (coinjoin-type) protocol is able to make eventual
progress, by having a candidate multiparty transaction either eventually
confirm or become conflicted with something that has been confirmed, in
which case the double-spend information could be used to start a new
coinjoin round with fewer participants. The concern Antoine and Greg have
brought up is that non-rbf transactions can persist in the mempool
~indefinitely (at a low feerate and not subject to replacement) and
interfere with progress being made in a coinjoin protocol.
However, it seems to me that similar problems exist for such a protocol
even in a fullrbf world, as we understand that term today. I mentioned the
ability for rbf “pinning” to interfere with relay of the multiparty
transaction (even if the conflicting transaction signals for RBF – a set of
large but low feerate conflicting transactions can persist in the mempool
and make it difficult for the coinjoin transaction from confirming, at
least without attaching a very large fee); and as Greg mentioned in a
followup, the BIP 125 rule to only permit 100 transactions to be removed
from the mempool at a time during a replacement can also be used to pin a
coinjoin protocol in the same way as a non-rbf transaction today. It seems
to me that what these multiparty protocols actually need is some sort of
"maximal rbf" network policy: a way to guarantee that a transaction which
should be desirable for a miner to mine would always get to a miner and
considered for inclusion in a block, no matter what the state of node’s
mempools on the network.
While that sounds like a reasonable thing to want on its face (and worth
working on), it's not how opt-in RBF works today, nor is it how transaction
relay has ever conceptually worked. We have not, thus far, been able to
come up with a total ordering on transaction desirability. Moreover, due
to all the DoS issues that exist with transaction relay, there are plenty
of seemingly legitimate ways to construct transactions that would not relay
well on the network. Relay has only ever been a best-efforts concept,
where we carve out a small subset of the entire transaction universe for
which we try to optimize propagation. The idea behind this approach is
that if every use case we can come up with has some way to achieve its
goals using transactions that should (eventually) be able to relay, then
users wouldn’t have much demand for transactions that would deviate from
the supported policies, and we therefore shouldn’t need to worry too much
about incentive compatibility concerns when it comes to transaction types
that wouldn’t relay at all, even if they are high feerate. (And when those
situations arise where the standard transactions do not accommodate some
needed use case, developers typically work to define a policy that is
compatible with our anti-DoS goals to support such use cases, such as with
the recent proposal for version=3 transactions [2].)
BIP 125's RBF rules themselves were an effort to carve out just a subset of
situations where a transaction should evict conflicting ones -- it was not
a design that anyone thought would ensure that all replacements which
"should" be mined would always propagate. And I don't believe that we know
how to design policy rules that would achieve the goals of this kind of
multiparty protocol in a DoS resistant way, today. Along those lines, I
would point out that even the BIP 125 design itself is not entirely
incentive compatible, in that it is possible to construct a replacement
transaction that would evict transactions which would be preferable to be
included in a block! [3] (This has been known for years, but fixing this
has proven difficult, and the only way to fix it that I’m aware of would be
to make BIP 125 RBF even more restrictive than it is today. I do think this
is something that needs to be worked on.)
Given the limitations of RBF as we have it today, it appears to be
incorrect that a fullrbf network policy would solve the problems Antoine
raised. And so absent any other examples, it does not seem to me that
fullrbf solves any problems for RBF users, who are already free to choose
to subject their transactions to BIP 125’s RBF policy. From this
perspective, "enabling fullrbf" is really just taking away user choice to
opt a transaction into a non-replacement policy regime.
I think we should ask, then, whether it is reasonable on its face that
users might want to opt-in to a non-replacement policy? Or in other words,
is it reasonable for a user to mark a transaction as non-replaceable and
have that indication be enforced by the network? Note that these are two
different questions: you could imagine a world where fullrbf is a dominant
policy, but users still use the BIP 125 signaling method to indicate, in an
unenforced way, their intention to not replace a transaction. This might
give useful information to the network or the recipient for how to interact
with such a transaction.
And I think that it's entirely possible that users would continue to use
the BIP 125 signaling to indicate that they do not intend to replace a
transaction. For better or worse, this might be because zeroconf services
continue to differentiate their behavior based on such a signal (possibly
in conjunction with other factors), or it could be because there are other
behaviors that could be utilized more effectively if the transaction
originator has made such a signal, such as the recipient chaining an
unconfirmed transaction as a way to bump the fee (CPFP) [4].
If it were to be the case that users continued to use BIP 125-style
signaling to indicate that they do not plan to replace a transaction, would
that be harmful to the network? This is not something we can stop in our
policy rules (short of censoring such transactions, an obviously bad
idea). I think network actors can always do things that we might think are
harmful for the network, but that doesn’t mean that there are no legitimate
use cases for the tools that such actors might be using. Just because
someone might use some policy to adopt a zeroconf model, doesn’t mean that
others aren’t using the same policy to achieve benign ends (such as better
CPFP behavior).
Moreover, while users might attempt to exploit services that offer zeroconf
or other differentiated behavior to non-replacement signaling transactions,
they also might not -- I think predicting user behavior in this way (and
specifically predicting the complexities of what a business might do and
whether users might try to subvert it) is beyond the scope of what we can
do as protocol developers. Instead, I think we can try to answer a
different question: if a group of users were to want the ability to opt-in
to a non-replacement policy regime, is that a technically sound option for
us to have on the network and enforce in software? Specifically, does that
interfere with having a sensible anti-DoS mempool acceptance algorithm, or
interfere with other protocols on the network, or necessarily run counter
to the interests of miners or node operators?
And I think the answer to that question, in looking at the difference
between opt-in RBF and fullrbf, is no: offering the ability to opt-in to a
non-replacement regime for transactions doesn't introduce any fundamental
issues with software or network policy or other protocols. In a world
where we only had fullrbf, I could imagine at some point down the road
proposing a non-replacement signal myself, because the complexities around
transaction chains (and pinning) are more complex for the RBF case than for
the non-RBF case (and BIP 125 is not always incentive compatible to begin
with!). Conceptually, this is no different to me than the version=3
transaction policy proposal that has been advancing, if we think of it as a
special set of restrictions on transactions designed to accommodate a
particular use case.
Philosophically, I think we should be looking to add non-interfering use
cases to what the network supports.
To those who argue for making fullrbf a default policy on the network (or
even just offering a flag for users to enable fullrbf), I pose this
hypothetical: suppose we deploy the v3 transaction policy proposal (which I
hope will happen in the near future). That policy would restrict the ways
that outputs of a v3 transaction can be spent while the transaction is
unconfirmed, including by limiting the number and size of descendants that
such a transaction can have, and limiting the types of unconfirmed
ancestors that can be included. Suppose in a few years someone proposes
that we add a "-disable_v3_transaction_enforcement" flag to our software,
to let users decide to turn off those policy restrictions and treat v3
transactions the same as v2, for all the same reasons that could be argued
today with fullrbf: miners might earn more revenue if we allowed multiple
descendant v3 transactions; it's illogical for the recipient of a v3
transaction to believe what is a fundamentally unenforceable promise of a
sender to not issue more high value children that descend from an
unconfirmed transaction; it's inappropriate for Bitcoin Core to dictate
policy on the network and we should honor user choice to turn off that flag
if that’s what users want; if users are relying on v3’s policy restrictions
for security then that is an unstable model and we should assume it will
get broken[5].
It’s obvious to me that adding a flag to disable v3 policy would be
subversive to making the lightning use case for v3 transactions work. And
so my response to such a hypothetical proposal would be to argue that no,
we should not enable users to disable this policy, because as long as that
policy is just optional and working for those who want it, it shouldn’t
harm anyone that we offer a tighter set of rules for a particular use
case. Adding a way to bypass those rules is just trying to break someone
else’s use case, not trying to add a new one. We should not wield
"incentive compatibility" as a bludgeon for breaking things that appear to
be working and not causing others harm.
I think this is exactly what is happening with fullrbf.
In comparing v3 transaction policy with opting out of transaction
replacement, there is of course one significant difference that I have
ignored thus far: I think the real difference is an opinion about whether
non-replacement transactions that are being used today are, overall, bad
for Bitcoin, and whether lightning’s use of v3 transactions in the future
would be bad for Bitcoin. If you think that zeroconf is unequivocally bad,
and that no one will be able to plausibly construct a case that lightning
is bad, then that qualitative judgment might sway you to not worrying about
the philosophical issues I've raised above, because these situations can be
distinguished.
However I am not personally willing to say that I think, overall,
non-rbf-signaling transactions in use on the network today are bad for
Bitcoin (or that fullrbf is definitely good – BIP 125’s rbf rules are
something we’ve been trying to improve upon for years, with little
success). Nor am I convinced that someone couldn’t put together a cogent
argument for lightning being bad for Bitcoin, because of its reliance on
relay policies that are difficult to design and impossible to guarantee as
part of its security model. So I choose instead to merely make a judgment
that seems more factually verifiable, which is that non-replacement is a
policy widely in use on the network today, and we largely don't have reason
to think (as far as I know!) that the network is seeing a lot of
transactions that would violate that policy.
If it did turn out that users were commonly signaling non-replacement, but
then signing and trying to relay doublespends, then I think that would be a
very good reason for Bitcoin Core to adopt fullrbf to reflect the reality
of what is happening. In the meantime, I think it makes more sense to say
that because we have BIP 125, there seems to be no need for users to signal
one way and behave another, and therefore there is no need to offer
software that might break a policy that is working well for some users.
Other software projects might choose differently, and it is after all a
permissionless network, so if this is in fact an unstable equilibrium that
will not last, then presumably someday it will be apparent it is not
working and we’ll abandon it. But I think the philosophy of transaction
relay policy in Bitcoin Core should be to support disparate use cases in
order to try to make everything work better, rather than break things
prematurely because we guess others will break them eventually anyway.
For those that have read this long email and still favor a fullrbf network
policy (or even just the ability for users to be able to turn on fullrbf
for themselves), I’d ask for thoughts on the following questions, which
have guided my thinking on this:
Does fullrbf offer any benefits other than breaking zeroconf business
practices? If so, what are they?
Is it reasonable to enforce BIP 125's rbf rules on all transactions, if
those rules themselves are not always incentive compatible?
If someone were to propose a command line option that breaks v3 transaction
relay in the future, is there a logical basis for opposing that which is
consistent with moving towards fullrbf now?
Cheers,
Suhas
[1]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
[2]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
[3] This is because under the BIP 125 rules, the feerate of the replacement
transaction is not compared to the individual feerates of all transactions
being evicted – we just compare feerates with the transactions that are
directly in conflict (and not their descendants). So it’s possible for a
transaction that would evict 2 or more transactions to have a higher
feerate than the direct conflicts, and higher total fee than the set being
evicted, but have a lower feerate (eg if it is larger) than that of some
subset of the set of transactions being evicted.
[4] Chaining unconfirmed transactions when the sender might RBF the parent
is far riskier than if the sender indicates they don't plan to do so
(chaining onto an RBF transaction creates pinning issues for the sender,
and risks having the child wiped out if the parent is replaced), so I think
this is a concrete reason why signaling that a transaction won’t be
replaced could be useful.
[5] This is a subtle point. I don’t think v3 transactions create an
unreasonable security assumption for the use case it is being designed for.
However, I don’t think anyone could rule out the possibility that someone
could adopt a usage pattern for v3 transactions that subverts the intent of
this policy. For example, if users started using v3 transactions for all
their payments, then the limitations on the number of descendants could
directly interfere with CPFP by a recipient, and someone could argue that
we should break the policy in order to allow for this hypothetical
behavior. I think this is a similar form of argument as saying that
zeroconf practices + BIP 125 create an incentive to double-spend non-rbf
signaling transactions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20221031/82fe4bab/attachment-0001.html>

2023-06-07 23:15:53

- reply

📅 Original date posted:2022-10-27
📝 Original message:I have more to say on this broader topic, but since you've brought up this
particular example I think it's worth commenting:
On Thu, Oct 27, 2022 at 1:23 PM Anthony Towns via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Is that true? Antoine claims [1] that opt-in RBF isn't enough to avoid
> a DoS issue when utxos are jointly funded by untrusting partners, and,
> aiui, that's the main motivation for addressing this now.
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>
> The scenario he describes is: A, B, C create a tx:
>
> inputs: A1, B1, C1 [opts in to RBF]
> fees: normal
> outputs:
> [lightning channel, DLC, etc, who knows]
>
> they all analyse the tx, and agree it looks great; however just before
> publishing it, A spams the network with an alternative tx, double
> spending her input:
>
> inputs: A1 [does not opt in to RBF]
> fees: low
> outputs: A
>
> If A gets the timing right, that's bad for B and C because they've
> populated their mempool with the 1st transaction, while everyone else
> sees the 2nd one instead; and neither tx will replace the other. B and
> C can't know that they should just cancel their transaction, eg:
>
> inputs: B1, C1 [opts in to RBF]
> fees: 50% above normal
> outputs:
> [smaller channel, refund, whatever]
>
> and might instead waste time trying to fee bump the tx to get it mined,
> or similar.
>
> What should folks wanting to do coinjoins/dualfunding/dlcs/etc do to
> solve that problem if they have only opt-in RBF available?
>
I think this is not a real example of a DoS vector that is available
because we support non-rbf signaling transactions. Even in a world where
all transactions are replaceable, person A could double-spend their input
in a way that is annoying for B and C. For instance, the double-spend
could be low-feerate and large, and effectively pin any attempt to replace
it. Or it could be higher feerate and confirm and B/C have to start all
over. Or, A could stall things in the signing phase and B/C have to figure
out when to give up on the channel with A.
So I find this example to be unconvincing. Are there any other examples
where having a non-replacement policy for some transactions causes problems
for protocols people are trying to build?
Thanks,
Suhas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20221027/511dc909/attachment-0001.html>

2023-06-07 23:14:08

- reply

📅 Original date posted:2022-10-04
📝 Original message:(Apologies for the double-post -- I'm resending this message to the list
with much of the quoted text trimmed, because my first message was placed
in the moderation queue for being too large)
Hi,
Thanks for sharing your thoughts on packaged transaction relay.
The sole objective, as expressed in the OP proposal, is to:
"Propagate transactions that are incentive-compatible to mine, even if they
> don't meet minimum feerate alone."
I actually do think there are additional goals we should include in any
protocol change involving transaction relay, such as ensuring that we
minimize bandwidth waste as much as possible (as I mentioned in a previous
message in this thread).
While I understand your proposal seeks to improve on an idea of static
packages in favor of dynamic package construction based on knowledge a node
should have of its peers, I think the main drawback of your proposal is
that it doesn't take into account the complexities of what a peer's
"minimum feerate" might mean. The consequence of this is that it's not
generally possible for a node to accurately guess whether a given
transaction should be sent in a package to a given peer, or not, and so in
addition to any "packaged transaction relay" mechanism that is implemented
by a transaction announcer, we'd still need to add protocol support for a
receiving peer to retrieve a package as well.
First of all, a node's feerate is a dynamic value. BIP 133 allows for
nodes to send feefilter messages at any time during the lifetime of a peer
connection. If we were to compare the feerate of ancestors of a relayed
transaction to the feerate in place at a peer as indicated by feefilter
messages, and use that determine whether those ancestors would have been
successfully relayed or not, then doing so accurately would seem to require
caching relay success for each transaction, for each peer, at the time such
transaction is relayed (or perhaps caching feefilter values that are
received from a peer?). This seems likely to be undesirable, and, at any
rate, is more complex than merely comparing a pair of feerates.
But even more fundamental than feerates being dynamic is that feerate
itself is not a well-defined concept when applied to arbitrary transaction
(sub-)graphs, and this is at the crux of the difficulty in coming up with
proposals that would meet the objective of ensuring that transactions which
are incentive-compatible to mine all get relayed successfully across the
network. Here are a couple examples that illustrate this:
- Let A be a low feerate transaction (below any peer's minimum feerate).
Let B and C be descendants of A (but are otherwise unrelated). Suppose
these transactions are relayed to a node in the order A, B, C. In the
algorithm you proposed, I believe the determination for whether C should be
announced to a given peer as a package (A, C) or as a singleton would
either (1) depend on whether the package (A, B) was sufficient to meet the
peer's feerate, or (2) waste bandwidth by engaging in packaged relay
whenever A was already successfully relayed as part of a package. Both of
these approaches seem undesirable.
- Let A be a high fee, but low feerate transaction. Suppose B is a
transaction that conflicts with A, has a high feerate, but lower total
fee. In this situation, two different nodes that learned of these two
transactions in opposite order [(A, B) vs (B, A)] might be expected to have
differing mempools -- this at least would be the case in the BIP 125
algorithm (which requires that both feerate and total fee must increase
when replacing an existing transaction), and at any rate it's not obvious
from the information given which would be more optimal to include in a
block, as that depends on factors that go beyond just these two
transactions. Suppose further that a new transaction C is relayed on the
network, which is a child of B and very high feerate, such that B + C have
higher fee and higher feerate than A, when taken together. In this case
we'd want our relay protocol to ensure that even nodes which saw A first
should still have a chance to consider transaction C, but the packaging
design you propose (which would compare transaction B's feerate to the
peer's, and conclude packaging is unnecessary because B's feerate might
already exceed the peer's feerate) would not facilitate this.
To summarize, the point I'd make from these examples is that we should not
expect that "feerate" (whatever that means) alone will be a sufficient
predictor of what is in our peer's mempools. So while there may be some
situations where a transaction relayer might be able to usefully package up
a transaction with its dependencies (perhaps in narrowly defined
situations), there will also always be situations where this isn't
possible, and what I conclude from that is that it should be helpful to add
to the protocol some way for the recipient of a transaction to request the
dependencies directly.
Taken together, I roughly understand Gloria's efforts here to be a
combination of these two approaches: add some amount of packaged
transaction relay for simple cases (ie where the transaction graph has been
sufficiently narrowed, to minimize bandwidth waste while reducing latency),
and also allow for a fallback mechanism where the recipient of a
transaction can efficiently retrieve dependencies. There seems to be a
tradeoff involving latency, bandwidth, and robustness of these two
approaches (and maybe also implementation complexity), so I think it's
natural to expect that it will take some discussion and understanding of
what practices are common on the network and what behaviors wallet or other
software might adopt, given potential protocol changes, to figure out how
best to balance these ideas.
On Wed, Jun 8, 2022 at 6:43 PM <eric at voskuil.org> wrote:
> Hi Suhas/Gloria,
>
> Good questions. I've started a new thread because it became something
> else...
>
> Various ideas about packaging seem to be focused on the idea of an atomic
> message that is gossiped around the network like a transaction or block.
> From my perspective that seems to create a set of problems without good
> solutions, and it is not a proper analogy to those atomic structures. It
> may be worth taking the time to step back and take a close look at the
> underlying objective.
>
> The sole objective, as expressed in the OP proposal, is to:
>
> "Propagate transactions that are incentive-compatible to mine, even if
> they don't meet minimum feerate alone."
>
> Effectively producing this outcome with an atomic packaging approach while
> at the same time maintaining network invariants seems unlikely, if not
> impossible.
>
> Fees:
>
> A node knows what fee rate a peer will accept, and announces individual
> txs that satisfy peer.feerate. Similarly a node knows its own feerate, and
> SHOULD drop any peer that announces txs that do not satisfy node.feerate.
>
> Orphans:
>
> A node MAY drop a peer that announces txs that the node sees as orphans
> against its DAG. It SHOULD drop the orphan tx and MAY request missing
> ancestors. Presumably after some amount of time connected to peer, node
> does not expect to see any more orphans from that peer, so these choices
> could evolve with the channel. However, the design that can only consider
> each tx in isolation will continue to cause orphan announcements on the
> channel. A below peer.feerate tx does not get announced to peer, and later
> a descendant high peer.feerate does get announced to the peer - as an
> orphan.
>
> BIP133 (feefilter):
>
> "There could be a small number of edge cases where a node's mempool min
> fee is actually less than the filter value a peer is aware of and
> transactions with fee rates between these values will now be newly
> inhibited."
>
> https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki
>
> Whether the problem is "small" or not depends on the disparity between
> node fee rates, which is not a matter of protocol. This is an existing
> problem that can and should be dealt with in packaging, as part of the
> above objective.
>
> Packaged Transaction Relay:
>
> One might instead think of packaging as a per-connection function,
> operating over its transaction (input->output) DAG and the feerate of its
> own node and that of the peer. Logically a "package" is nothing more than a
> set of transactions (optimized by announcement). Only a node can
> effectively determine the packaging required by each of its peers, since
> only the node is aware of peer.feerate.
>
> The only way to avoid dead-ending packages (including individual
> transactions, as is the objective) is for a node to package txs for each
> peer. The origination of any package is then just a wallet peer doing what
> a node does - packaging transactions that satisfy peer.feerate (i.e. that
> of its node).
>
> Current transaction relay (txB->txA):
> ===============================
> Node0
> txA.feerate > node.feerate, and not orphaned (accept txA)
> txA.feerate > peer1.feerate (announce txA to peer1)
> txA.feerate < peer2.feerate (do not announce txA to peer2)
> -----
> txB.feerate > node.feerate (accept txB)
> txB.feerate > peer1.feerate (announce txB to peer1)
> txB.feerate > peer2.feerate (announce txB to peer2)
>
> Node1
> Sees/accepts txA and txB.
>
> Node2
> Never sees txA, sees/rejects txB (as an orphan).
>
> Packaged transaction relay (txB->txA):
> ===============================
> Node0
> txA.feerate > node.feerate, and not orphaned (accept txA)
> txA.feerate > peer1.feerate (announce txA to peer1)
> txA.feerate < peer2.feerate (do not announce txA to peer2)
> -----
> txB.feerate > node1.feerate (accept txB)
> txB.feerate > peer1.feerate (announce txB to peer1)
> txB.feerate > peer2.feerate (do not announce txB to peer2) <== avoid
> predictable orphan
> txA.feerate + txB.feerate > peer2.feerate (announce pkg(A, B) to peer2) <=
> create minimal package
>
> Node1
> Sees/accepts txA and txB.
>
> Node2
> pkg(A, B) > node2.feerate (accept txA, txB)
> txA.feerate > peer3.feerate (announce txA to peer3)
> txB.feerate > peer3.feerate (announce txB to peer3)
>
> Sees/accepts pkg(A, B).
>
> Node3
> Sees/accepts txA and txB. <= avoided unnecessary packaging
>
> Summary:
>
> In this design, any node that receives an announcement for a pkg (or tx)
> later determined to be less than node.feerate SHOULD drop the announcing
> peer. Unlike with existing tx relay, a node can become "current" and
> subsequently see few if any tx or pkg orphans, and MAY at some point decide
> to drop any peer that announces one. Notice that packages are created
> dynamically, and any package that doesn't need to be grouped gets trimmed
> down to individual transactions. Furthermore any tx that is "stuck" can be
> freed by simply sending another tx. The nodes at which the tx has become
> stuck will just package it up and relay it to peers. In other words, there
> is no impact on wallet implementation apart from raising the aggregate fee
> using a descendant transaction.
>
> This is barely a protocol change - it's primarily implementation. All that
> should be required is an additional INV element type, such as
> MSG_TX_PACKAGE.
>
> Additional constraints:
>
> * All elements of MSG_TX_PACKAGE in one INV message MUST to be of the same
> package.
> * A package MUST must define a set that can be mined into one block
> (size/sigops constraint).
> * A package SHOULD not contain confirmed txs (a race may cause this).
> * A package MUST minimally satisfy peer.feerate.
> * A partial tx order, as in the manner of the block.txs ordering, MUST be
> imposed.
> * A node SHOULD drop a peer that sends a package (or tx) below
> node.feerate.
> * A node MAY drop a peer that sends a non-minimal package according to
> node.feerate.
>
> The partial ordering of block.txs introduces an ordering constraint that
> precludes full parallelism in validating input attachment. This is an
> implementation artifact that made its way into consensus. However in the
> case of packaging, the set of txs is not presumed to be valid under the
> proof of work DoS guard. As such constraints should minimize the
> work/traffic required to invalidate the message. The partial order
> constraint ensures that the DAG can be built incrementally, dropping the
> attempt (and peer as desired) as soon as the first orphan is discovered. As
> a result the network traffic and work required is not materially different
> than with tx relay, with two exceptions.
>
> These are the two central aspects of this approach (Avoiding Predictable
> Orphans and Creating Minimal Packages). These are graph search algorithms,
> some basic computer science. Minimality requires only that the package does
> not introduce txs that are not necessary to reach the peer.feerate (as
> these can always be packaged separately). It does not require that nodes
> all generate the same packages. It does not require negotiation, package
> identity, cryptography, or hashing. As a graph search it should be O(n)
> where n is the unconfirmed ancestry of the package, but should typically be
> much lower, if not a single step.
>
> Sufficiently-low-fee nodes will see only single txs. Moderate-fee nodes
> may cause partial breakup of packages. Sufficiently high fee nodes will
> cause peers (having received and completed the acceptance of a tx/pkg with
> pkg.feerate < peer.feerate) to navigate from each tx/package external input
> until reaching txs above peer.feerate, or confirmed (both of which the peer
> is presumed to already have). If the pkg.feerate is sufficiently high to
> connect all external inputs to the intervening txs, they are added to the
> package and it is announced to the high fee peer. Note that the individual
> tx.feerate > peer.feerate is insufficient to ensure that the peer should
> have the tx, as there may be ancestor txs that do not, and for which the tx
> was insufficient to cause them to be packaged. So a non-caching algorithm
> must be able to chase each package external input to a confirmed tx (or
> cache the unconfirmed ancestry fee rate at each tx). Note that fee rates
> are not directly additive, both size/weight and fee are required for
> summation (and aggregate sigops should be considered).
>
> This makes no assumptions about current implementations. The design would
> call for maintenance of a transaction (input->output) DAG with tx.feerate
> on each tx. This could be the unconfirmed tx graph (i.e. "memory pool")
> though it does not require maintenance of anything more than the parameters
> necessary to confirm a set of validated txs within a block. It is very
> reasonable to require this of any participating node. A simple version
> negotiation can identify a package-accepting/sending nodes.
>
> I have thought about this for some time, but have not implemented either
> the graph search, source code, or BIP. Just wrote this off the top of my
> head. So I am sure there are some things I have incorrect or failed to
> consider. But I think it's worth discussing it at this point.
>
> e
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20221004/7cac036f/attachment-0001.html>

2023-06-07 23:10:19

- reply

📅 Original date posted:2022-06-08
📝 Original message:Hi,
Thanks again for your work on this!
One question I have is about potential bandwidth waste in the case of nodes
running with different policy rules. Here's my understanding of a scenario
I think could happen:
1) Transaction A is both low-fee and non-standard to some nodes on the
network.
2) Whenever a transaction T that spends A is relayed, new nodes will send
INV(PKGINFO1, T) to all package-relay peers.
3) Nodes on the network that have implemented package relay, but do not
accept A, will send getdata(PKGINFO1, T) and learn all of T's unconfirmed
parents (~32 bytes * number of parents(T)).
4) Such nodes will reject T. But because of transaction malleability, and
to avoid being blinded to a transaction unnecessarily, these nodes will
likely still send getdata(PKGINFO1, T) to every node that announces T, in
case someone has a transaction that includes an alternate set of parent
transactions that would pass policy checks.
Is that understanding correct? I think a good design goal would be to not
waste bandwidth in non-adversarial situations. In this case, there would
be bandwidth waste from downloading duplicate data from all your peers,
just because the announcement doesn't commit to the set of parent wtxids
that we'd get from the peer (and so we are unable to determine that all our
peers would be telling us the same thing, just based on the announcement).
Some ways to mitigate this might be to: (a) include a hash (maybe even just
a 20-byte hash -- is that enough security?) of the package wtxids (in some
canonical ordering) along with the wtxid of the child in the initial
announcement; (b) limit the use of v1 packages to transactions with very
few parents (I don't know if this is reasonable for the use cases we have
in mind).
Another point I wanted to bring up is about the rules around v1 package
validation generally, and the use of a blockhash in transaction relay
specifically. My first observation is that it won't always be the case
that a v1 package relay node will be able to validate that a set of package
transactions is fully sorted topologically, because there may be
(non-parent) ancestors that are missing from the package and the best a
peer can validate is topology within the package -- this means that a peer
can validly (under this BIP) relay transaction packages out of the true
topological sort (if all ancestors were included).
This makes me wonder how useful this topological rule is. I suppose there
is some value in preventing completely broken implementations from staying
connected and so there is no harm in having the rule, but perhaps it would
be helpful to add that nodes SHOULD order transactions based on topological
sort in the complete transaction graph, so that if missing-from-package
ancestors are already known by a peer (which is the expected case when
using v1 package relay on transactions that have more than one generation
of unconfirmed ancestor) then the remaining transactions are already
properly ordered, and this is helpful even if unenforceable in general.
The other observation I wanted to make was that having transaction relay
gated on whether two nodes agree on chain tip seems like an overly
restrictive criteria. I think an important design principle is that we
want to minimize disruption from network splits -- if there are competing
blocks found in a small window of time, it's likely that the utxo set is
not materially different on the two chains (assuming miners are selecting
from roughly the same sets of transactions when this happens, which is
typical). Having transaction relay bifurcate on the two network halves
would seem to exacerbate the difference between the two sides of the split
-- users ought to be agnostic about how benign splits are resolved and
would likely want their transactions to relay across the whole network.
Additionally, use of a chain tip might impose a larger burden than is
necessary on software that would seek to participate in transaction relay
without implementing headers sync/validation. I don't know what software
exists on the network, but I imagine there are a lot of scripts out there
for transaction submission to the public p2p network, and in thinking
about modifying such a script to utilize package relay it seems like an
unnecessary added burden to first learn a node's tip before trying to relay
a transaction.
Could you explain again what the benefit of including the blockhash is? It
seems like it is just so that a node could prioritize transaction relay
from peers with the same chain tip to maximize the likelihood of
transaction acceptance, but in the common case this seems like a pretty
negligible concern, and in the case of a chain fork that persists for many
minutes it seems better to me that we not partition the network into
package-relay regimes and just risk a little extra bandwidth in one
direction or the other. If we solve the problem I brought up at the
beginning (of de-duplicating package data across peers with a
package-wtxid-commitment in the announcement), I think this is just some
wasted pkginfo bandwidth on a single-link, and not across links (as we
could cache validation failure for a package-hash to avoid re-requesting
duplicate pkginfo1 messages).
Best,
Suhas
On Tue, Jun 7, 2022 at 1:57 PM Gloria Zhao via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi Eric, aj, all,
>
> Sorry for the delayed response. @aj I'm including some paraphrased points
> from our offline discussion (thanks).
>
> > Other idea: what if you encode the parent txs as a short hash of the
> wtxid (something like bip152 short ids? perhaps seeded per peer so
> collisions will be different per peer?) and include that in the inv
> announcement? Would that work to avoid a round trip almost all of the time,
> while still giving you enough info to save bw by deduping parents?
>
> > As I suggested earlier, a package is fundamentally a compact block (or
> > block) announcement without the header. Compact block (BIP152)
> announcement
> > is already well-defined and widely implemented...
>
> > Let us not reinvent the wheel and/or introduce accidental complexity. I
> see
> > no reason why packaging is not simply BIP152 without the 'header' field,
> an
> > updated protocol version, and the following sort of changes to names
>
> Interestingly, "why not use BIP 152 shortids to save bandwidth?" is by far
> the most common suggestion I hear (including offline feedback). Here's a
> full explanation:
>
> BIP 152 shortens transaction hashes (32 bytes) to shortids (6 bytes) to
> save a significant amount of network bandwidth, which is extremely
> important in block relay. However, this comes at the expense of
> computational complexity. There is no way to directly calculate a
> transaction hash from a shortid; upon receipt of a compact block, a node is
> expected to calculate the shortids of every unconfirmed transaction it
> knows about to find the matches (BIP 152: [1], Bitcoin Core: [2]). This is
> expensive but appropriate for block relay, since the block must have a
> valid Proof of Work and new blocks only come every ~10 minutes. On the
> other hand, if we require nodes to calculate shortids for every transaction
> in their mempools every time they receive a package, we are creating a DoS
> vector. Unconfirmed transactions don't need PoW and, to have a live
> transaction relay network, we should expect nodes to handle transactions at
> a high-ish rate (i.e. at least 1000s of times more transactions than
> blocks). We can't pre-calculate or cache shortids for mempool transactions,
> since the SipHash key depends on the block hash and a per-connection salt.
>
> Additionally, shortid calculation is not designed to prevent intentional
> individual collisions. If we were to use these shortids to deduplicate
> transactions we've supposedly already seen, we may have a censorship
> vector. Again, these tradeoffs make sense for compact block relay (see
> shortid section in BIP 152 [3]), but not package relay.
>
> TLDR: DoSy if we calculate shortids on every package and censorship vector
> if we use shortids for deduplication.
>
> > Given this message there is no reason
> > to send a (potentially bogus) fee rate with every package. It can only
> be
> > validated by obtaining the full set of txs, and the only recourse is
> > dropping (etc.) the peer, as is the case with single txs.
>
> Yeah, I agree with this. Combined with the previous discussion with aj
> (i.e. we can't accurately communicate the incentive compatibility of a
> package without sending the full graph, and this whole dance is to avoid
> downloading a few low-fee transactions in uncommon edge cases), I've
> realized I should remove the fee + weight information from pkginfo. Yay for
> less complexity!
>
> Also, this might be pedantic, but I said something incorrect earlier and
> would like to correct myself:
>
> >> In theory, yes, but maybe it was announced earlier (while our node was
> down?) or had dropped from our mempool or similar, either way we don't have
> those txs yet.
>
> I said "It's fine if they have Erlay, since a sender would know in advance
> that B is missing and announce it as a package." But this isn't true since
> we're only using reconciliation in place of flooding to announce
> transactions as they arrive, not for rebroadcast, and we're not doing full
> mempool set reconciliation. In any case, making sure a node receives the
> transactions announced when it was offline is not something we guarantee,
> not an intended use case for package relay, and not worsened by this.
>
> Thanks for your feedback!
>
> Best,
> Gloria
>
> [1]:
> https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki#cmpctblock
> [2]:
> https://github.com/bitcoin/bitcoin/blob/master/src/blockencodings.cpp#L49
> [3]:
> https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki#short-transaction-id-calculation
>
> On Thu, May 26, 2022 at 3:59 AM <eric at voskuil.org> wrote:
>
>> Given that packages have no header, the package requires identity in a
>> BIP152 scheme. For example 'header' and 'blockhash' fields can be replaced
>> with a Merkle root (e.g. "identity" field) for the package, uniquely
>> identifying the partially-ordered set of txs. And use of 'getdata' (to
>> obtain a package by hash) can be eliminated (not a use case).
>>
>> e
>>
>> > -----Original Message-----
>> > From: eric at voskuil.org <eric at voskuil.org>
>> > Sent: Wednesday, May 25, 2022 1:52 PM
>> > To: 'Anthony Towns' <aj at erisian.com.au>; 'Bitcoin Protocol Discussion'
>> > <bitcoin-dev at lists.linuxfoundation.org>; 'Gloria Zhao'
>> > <gloriajzhao at gmail.com>
>> > Subject: RE: [bitcoin-dev] Package Relay Proposal
>> >
>> > > From: bitcoin-dev <bitcoin-dev-bounces at lists.linuxfoundation.org> On
>> > Behalf
>> > > Of Anthony Towns via bitcoin-dev
>> > > Sent: Wednesday, May 25, 2022 11:56 AM
>> >
>> > > So the other thing is what happens if the peer announcing packages to
>> us
>> > is
>> > > dishonest?
>> > >
>> > > They announce pkg X, say X has parents A B C and the fee rate is
>> garbage.
>> > But
>> > > actually X has parent D and the fee rate is excellent. Do we request
>> the
>> > > package from another peer, or every peer, to double check? Otherwise
>> > we're
>> > > allowing the first peer we ask about a package to censor that tx from
>> us?
>> > >
>> > > I think the fix for that is just to provide the fee and weight when
>> > announcing
>> > > the package rather than only being asked for its info? Then if one
>> peer
>> > makes
>> > > it sound like a good deal you ask for the parent txids from them,
>> dedupe,
>> > > request, and verify they were honest about the parents.
>> >
>> > Single tx broadcasts do not carry an advertised fee rate, however the'
>> > feefilter' message (BIP133) provides this distinction. This should be
>> > interpreted as applicable to packages. Given this message there is no
>> reason
>> > to send a (potentially bogus) fee rate with every package. It can only
>> be
>> > validated by obtaining the full set of txs, and the only recourse is
>> > dropping (etc.) the peer, as is the case with single txs. Relying on the
>> > existing message is simpler, more consistent, and more efficient.
>> >
>> > > >> Is it plausible to add the graph in?
>> > >
>> > > Likewise, I think you'd have to have the graph info from many nodes if
>> > you're
>> > > going to make decisions based on it and don't want hostile peers to be
>> > able to
>> > > trick you into ignoring txs.
>> > >
>> > > Other idea: what if you encode the parent txs as a short hash of the
>> wtxid
>> > > (something like bip152 short ids? perhaps seeded per peer so
>> collisions
>> > will
>> > > be different per peer?) and include that in the inv announcement?
>> Would
>> > > that work to avoid a round trip almost all of the time, while still
>> giving
>> > you
>> > > enough info to save bw by deduping parents?
>> >
>> > As I suggested earlier, a package is fundamentally a compact block (or
>> > block) announcement without the header. Compact block (BIP152)
>> > announcement
>> > is already well-defined and widely implemented. A node should never be
>> > required to retain an orphan, and BIP152 ensures this is not required.
>> >
>> > Once a validated set of txs within the package has been obtained with
>> > sufficient fee, a fee-optimal node would accept the largest subgraph of
>> the
>> > package that conforms to fee constraints and drop any peer that
>> provides a
>> > package for which the full graph does not.
>> >
>> > Let us not reinvent the wheel and/or introduce accidental complexity. I
>> see
>> > no reason why packaging is not simply BIP152 without the 'header' field,
>> an
>> > updated protocol version, and the following sort of changes to names:
>> >
>> > sendpkg
>> > MSG_CMPCT_PKG
>> > cmpctpkg
>> > getpkgtxn
>> > pkgtxn
>> >
>> > > > For a maximum 25 transactions,
>> > > >23*24/2 = 276, seems like 36 bytes for a child-with-parents package.
>> > >
>> > > If you're doing short ids that's maybe 25*4B=100B already, then the
>> above
>> > is
>> > > up to 36% overhead, I guess. Might be worth thinking more about, but
>> > maybe
>> > > more interesting with ancestors than just parents.
>> > >
>> > > >Also side note, since there are no size/count params,
>> >
>> > Size is restricted in the same manner as block and transaction
>> broadcasts,
>> > by consensus. If the fee rate is sufficient there would be no reason to
>> > preclude any valid size up to what can be mined in one block (packaging
>> > across blocks is not economically rational under the assumption that one
>> > miner cannot expect to mine multiple blocks in a row). Count is
>> incorporated
>> > into BIP152 as 'shortids_length'.
>> >
>> > > > wondering if we
>> > > >should just have "version" in "sendpackages" be a bit field instead
>> of
>> > > >sending a message for each version. 32 versions should be enough
>> right?
>> >
>> > Adding versioning to individual protocols is just a reflection of the
>> > insufficiency of the initial protocol versioning design, and that of the
>> > various ad-hoc changes to it (including yet another approach in this
>> > proposal) that have been introduced to compensate for it, though I'll
>> > address this in an independent post at some point.
>> >
>> > Best,
>> > e
>> >
>> > > Maybe but a couple of messages per connection doesn't really seem
>> worth
>> > > arguing about?
>> > >
>> > > Cheers,
>> > > aj
>> > >
>> > >
>> > > --
>> > > Sent from my phone.
>> > > _______________________________________________
>> > > bitcoin-dev mailing list
>> > > bitcoin-dev at lists.linuxfoundation.org
>> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220608/84353690/attachment-0001.html>

2023-06-07 18:28:10

- reply

📅 Original date posted:2021-01-19
📝 Original message:On Thu, Jan 14, 2021 at 1:46 AM Anthony Towns <aj at erisian.com.au> wrote:
> > ==Specification==
> > # A new disabletx message is added, [...]
> > # A node that has sent or received a disabletx message to/from a peer
> MUST NOT send any of these messages to the peer:
> > ## inv messages for transactions
> > [...]
> > # It is RECOMMENDED that a node that has sent or received a disabletx
> message to/from a peer not send any of these messages to the peer:
> > ## addr/getaddr
> > ## addrv2 (BIP 155)
>
> In particular, I think combining these doesn't make sense for:
>
> * nodes running with -blocksonly (that want to stay up to date with
> the blockchain, but don't care about txes)
> - not sending disabletx reduces their potential connectivity, if
> nodes are willing to accept more disabletx peers due to the lower
> resource usage
> - sending disabletx means they can't maintain their addr db and find
> other nodes to connect to
>
This is quite implementation specific, but the Bitcoin Core implementation
of -blocksonly already differs from what you would get if all outbound
connections were treated as block-relay-only peers; I initially tried to
make the -blocksonly setting behave the same as block-relay-only
connections and never relay transactions, but it turned out that Bitcoin
Core currently allows users running in -blocksonly mode to sometimes relay
transactions themselves (there's even a test for it [1]).
This isn't compatible with the design of disabletx, so I don't think it
could be used. I think that is okay, as I don't think we need to make
allowances right now to encourage the use of more -blocksonly nodes on the
network by prioritizing their existing outbound connections[2]. In the case
of block-relay-only connections, the next step I plan to propose in the
Bitcoin Core implementation is an increase in the number of inbound slots
to accommodate additional disabletx peers, to facilitate an eventual
increase in the number of outbound block-relay-only connections that
Bitcoin Core would initiate by default (and reduce potential concern about
overwhelming the network's capacity).
* non-listening nodes running with -connect to one/more preselected peers
> - they can't send disabletx generally because they want txes
> - they don't need addr information (since they only make connections
> to some known peers), and don't have many peers to relay addresses
> on to, so are essentially blackholes, so would like to disable
> addr relay for much the same reasons
>
While I think it's a valid use-case if someone wants to run nodes in this
way, I don't see why it has direct bearing on this discussion -- adding
protocol support for this use case is not incompatible with my disabletx
proposal; indeed, once we have worked out an addr relay protocol that
supports a peer telling another to not relay addresses, I would plan to use
it along with disabletx.
-----
There was some discussion about this proposal on IRC recently as well [3]
and this issue of mentioning addr-relay in a BIP about tx-relay seems to
have drawn some critique. I think if there were some other use case that
other developers have which would use disabletx (say, because the current
fRelay solution also is inadequate for those use cases today), yet would
want addr relay on these links, then that would be a good reason to drop
mention of it from the BIP and defer discussion of addr relay entirely to
some new future proposal. However, I'm not yet aware of such a use case,
as the -blocksonly one mentioned above does not actually fit (and nor do I
think that a -blocksonly mode where transactions are never sent but addr
messages are exchanged is necessary beyond how we already do that today, by
just sending fRelay=false).
Moreover, because there is no specification/BIP that governs how address
relay should work on the network, I think that if we just dropped mention
of addr-relay from the BIP, that software ought to just disable addr-relay
to nodes sending disabletx, anyway! Addr relay is currently purely local
policy, so I think it's reasonable to conclude that if the only known use
case of a peer sending disabletx is for a behavior that also would drop
addr messages, then optimizing software for that use case is logical and I
would propose that myself for how Bitcoin Core behaves. It seems polite to
mention this in the BIP as well, so that other software implementors have
the same understanding that I have for how this should work, until we have
an address relay protocol extension that governs it explicitly.
-----
Why not simply add a "disableaddr" type of message now? I am not opposed
to someone championing an addr relay protocol, but I personally think it is
premature. I'm not aware of any research or shared understanding of what
the goals of address relay are or ought to be, particularly as they pertain
to how addresses for obscure networks ought to propagate to nodes that may
or may not support those networks. Moreover, I'm not aware of any research
into what relay strategies make sense to achieve any particular relay goals
we might come up with. I imagine that we will someday propose a way for
software to communicate support for various network types (as defined in
BIP 155), and that will relate to the kinds of relay policies we expect
software on the network to be running. But I think any p2p message we add
now to simply "disableaddr" would be made redundant by a more extensive
protocol in the future, so I don't feel comfortable defending a proposal to
add such a message at this time, when it seems to be unnecessary for
today's use cases.
So to summarize my view on this BIP's recommendation against sending addr
messages to disabletx peers: if there is some software that someone plans
to deploy soon that would seek to use disabletx but would prefer addr relay
on these links, then I think that would cause me to re-evaluate this
proposal to figure out the best way to achieve both use cases. For now I
think the recommendation I've proposed still makes sense.
--Suhas
[1]
https://github.com/bitcoin/bitcoin/blob/f91587f050d9dceb45fe10129a76a4a9a060a09c/test/functional/p2p_blocksonly.py#L36
[2] Perhaps confusingly, running in -blocksonly mode is largely irrelevant
to the outbound peering of such a node, as it makes 10 outbound connections
by default, 8 of which are considered "full-relay" yet send fRelay=false to
not receive inbound transaction traffic, and 2 considered
"block-relay-only", also setting fRelay=false, but also not ever announcing
transactions on those links. I anticipate that in the future, Bitcoin Core
software running in -blocksonly mode will continue to treat those 8
"full-relay" connections the same as today, and just start sending the
disabletx message on the connections it considers to be truly
block-relay-only (the two existing connections, along with any additional
ones we later add).
[3]
https://bitcoin.jonasschnelli.ch/ircmeetings/logs/bitcoin-core-dev/2021/bitcoin-core-dev.2021-01-12-15.00.log.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210119/996bcadd/attachment-0001.html>

2023-06-07 18:28:08

- reply

📅 Original date posted:2021-01-06
📝 Original message:Hi,
I'm proposing the addition of a new, optional p2p message to allow peers to
communicate that they do not want to send or receive (loose) transactions
for the lifetime of a connection.
The goal of this message is to help facilitate connections on the network
over which only block-related data (blocks/headers/compact blocks/etc) are
relayed, to create low-resource connections that help protect against
partition attacks on the network. In particular, by adding a network
message that communicates that transactions will not be relayed for the
life of the connection, we ease the implementation of software that could
have increased inbound connection limits for such peers, which in turn will
make it easier to add additional persistent block-relay-only connections on
the network -- strengthening network security for little additional
bandwidth.
Software has been deployed for over a year now which makes such
connections, using the BIP37/BIP60 "fRelay" field in the version message to
signal that transactions should not be sent initially. However, BIP37
allows for transaction relay to be enabled later in the connection's
lifetime, complicating software that would try to distinguish inbound peers
that will never relay transactions from those that might.
This proposal would add a single new p2p message, "disabletx", which (if
used at all) must be sent between version and verack. I propose that this
message is valid for peers advertising protocol version 70017 or higher.
Software is free to implement this BIP or ignore this message and remain
compatible with software that does implement it.
Full text of the proposed BIP is below.
Thanks,
Suhas
---------------------------------------------------
<pre>
BIP: XXX
Layer: Peer Services
Title: Disable transaction relay message
Author: Suhas Daftuar <sdaftuar at chaincode.com>
Comments-Summary: No comments yet.
Comments-URI:
Status: Draft
Type: Standards Track
Created: 2020-09-03
License: BSD-2-Clause
</pre>
==Abstract==
This BIP describes a change to the p2p protocol to allow a node to tell a peer
that a connection will not be used for transaction relay, to support
block-relay-only connections that are currently in use on the network.
==Motivation==
For nearly the past year, software has been deployed[1] which initiates
connections on the Bitcoin network and sets the transaction relay field
(introduced by BIP 37 and also defined in BIP 60) to false, to prevent
transaction relay from occurring on the connection. Additionally, addr messages
received from the peer are ignored by this software.
The purpose of these connections is two-fold: by making additional
low-bandwidth connections on which blocks can propagate, the robustness of a
node to network partitioning attacks is strengthened. Additionally, by not
relaying transactions and ignoring received addresses, the ability of an
adversary to learn the complete network graph (or a subgraph) is reduced[2],
which in turn increases the cost or difficulty to an attacker seeking to carry
out a network partitioning attack (when compared with having such knowledge).
The low-bandwidth / minimal-resource nature of these connections is currently
known only by the initiator of the connection; this is because the transaction
relay field in the version message is not a permanent setting for the lifetime
of the connection. Consequently, a node receiving an inbound connection with
transaction relay disabled cannot distinguish between a peer that will never
enable transaction relay (as described in BIP 37) and one that will. Moreover,
the node also cannot determine that the incoming connection will ignore relayed
addresses; with that knowledge a node would likely choose other peers to
receive announced addresses instead.
This proposal adds a new, optional message that a node can send a peer when
initiating a connection to that peer, to indicate that connection should not be
used for transaction-relay for the connection's lifetime. In addition, without
a current mechanism to negotiate whether addresses should be relayed on a
connection, this BIP suggests that address messages not be sent on links where
tx-relay has been disabled.
==Specification==
# A new disabletx message is added, which is defined as an empty
message where pchCommand == "disabletx".
# The protocol version of nodes implementing this BIP must be set to
70017 or higher.
# If a node sets the transaction relay field in the version message to
a peer to false, then the disabletx message MAY also be sent in
response to a version message from that peer if the peer's protocol
version is >= 70017. If sent, the disabletx message MUST be sent prior
to sending a verack.
# A node that has sent or received a disabletx message to/from a peer
MUST NOT send any of these messages to the peer:
## inv messages for transactions
## getdata messages for transactions
## getdata messages for merkleblock (BIP 37)
## filteradd/filterload/filterclear (BIP 37)
## mempool (BIP 35)
# It is RECOMMENDED that a node that has sent or received a disabletx
message to/from a peer not send any of these messages to the peer:
## addr/getaddr
## addrv2 (BIP 155)
# The behavior regarding sending or processing other message types is
not specified by this BIP.
# Nodes MAY decide to not remain connected to peers that send this
message (for example, if trying to find a peer that will relay
transactions).
==Compatibility==
Nodes with protocol version >= 70017 that do not implement this BIP, and nodes
with protocol version < 70017, will continue to remain compatible with
implementing software: transactions would not be relayed to peers sending the
disabletx message (provided that BIP 37 or BIP 60 has been
implemented), and while
periodic address relay may still take place, software implementing this BIP
should not be disconnecting such peers solely for that reason.
Disabling address relay is suggested but not required by this BIP, to allow for
future protocol extensions that might specify more carefully how address relay
is to be negotiated. This BIP's recommendations for software to not relay
addresses is intended to be interpreted as guidance in the absence of any such
future protocol extension, to accommodate existing software behavior.
Note that all messages specified in BIP 152, including blocktxn and
getblocktxn, are permitted between peers that have sent/received a disabletx
message, subject to the feature negotiation of BIP 152.
==Implementation==
TBD
==References==
# Bitcoin Core has [https://github.com/bitcoin/bitcoin/pull/15759
implemented this functionality] since version 0.19.0.1, released in
November 2019.
# For example, see
https://www.cs.umd.edu/projects/coinscope/coinscope.pdf and
https://arxiv.org/pdf/1812.00942.pdf.
==Copyright==
This BIP is licensed under the 2-clause BSD license.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210106/a6651b31/attachment.html>

2023-06-07 18:26:58

- reply

📅 Original date posted:2020-09-22
📝 Original message:Hi,
I think the topic of how to improve transaction relay policy and fee
bumping is an important one that needs to be worked on, so I'm glad
this is a topic of discussion. However I am pretty skeptical of this
consensus change proposal:
The Sponsor Vector TXIDs must also be in the block the transaction is
> validated in, with no restriction on order or on specifying a TXID more
> than once.
That means that if a transaction is confirmed in a block without its
sponsor, the sponsor is no longer valid. This breaks a design principle
that has been discussed many times over the years, which is that once a
valid transaction is created, it should not become invalid later on unless
the inputs are double-spent. This principle has some logical consequences
that we've come to accept, such as transaction chains being valid across
small reorgs in the absence of malicious (double-spend) behavior.
I think that this principle is a useful one and that there should be a high
bar for doing away with it. And it seems to me that this proposal doesn't
clear that bar -- the fee bumping improvement that this proposal aims at is
really coming from the policy change, rather than the consensus change. But
if policy changes are the direction we're going to solve these problems, we
could instead just propose new policy rules for the existing types of
transaction chaining that we have, rather than couple them to a new
transaction type.
My understanding of the main benefit of this approach is that this allows
3rd parties to participate in fee bumping. But that behavior strikes me as
also problematic, because it introduces the possibility of 3rd party
griefing, to the extent that sponsor transactions in any way limit chains
of transactions that would be otherwise permitted. If Alice sends Bob some
coins, and Alice and Bob are both honest and cooperating, Mallory shouldn't
be able to interfere with their low-feerate transaction by (eg) pinning it
with a large transaction that "sponsors" it (ie a large transaction that is
just above the feerate of the parent, which prevents additional child
transactions and makes it more expensive to RBF).
This last issue of pinning could be improved in this proposal by requiring
that a sponsor transaction bring the effective feerate of its package up to
something which should be confirmed soon (rather than just being a higher
feerate than the tx it is sponsoring). However, we could also carve out a
policy rule just like that today, without any consensus changes needed, to
help with pinning (which is probably a good idea! I think this would be
useful work). So I don't think that approaches in that direction would be
unique to this proposal.
We allow one Sponsor to replace another subject to normal replacement
> policies, they are treated as conflicts.
This policy rule of allowing sponsor transactions to RBF each other also
seems problematic; that means that if Alice is paying Bob in a transaction
that is also sponsoring some other transaction (perhaps from Alice to
someone else), then Mallory can cause the transaction going to Bob to
become invalid by RBF bumping it and sponsoring the parent transaction
herself? Allowing 3rd parties to interfere with transactions between
others seems like a complex and undesirable design to introduce.
In summary: this proposal seems like a CPFP replacement, requiring many
policy rules along with a consensus change to be worked out to get right; I
think we could achieve largely the same effect by improving the current
policy rules to make CPFP work better without a consensus change. And
while what is unique about this proposal is that it allows for 3rd parties
to attach themselves to the transaction graph of other parties, I think
that is a complex interaction to introduce and has negative side effects as
well.
On Mon, Sep 21, 2020 at 12:27 PM Jeremy via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Responses Inline:
>
> Would it make sense that, instead of sponsor vectors
>> pointing to txids, they point to input outpoints? E.g.:
>>
>> 1. Alice and Bob open a channel with funding transaction 0123...cdef,
>> output 0.
>>
>> 2. After a bunch of state updates, Alice unilaterally broadcasts a
>> commitment transaction, which has a minimal fee.
>>
>> 3. Bob doesn't immediately care whether or not Alice tried to close the
>> channel in the latest state---he just wants the commitment
>> transaction confirmed so that he either gets his money directly or he
>> can send any necessary penalty transactions. So Bob broadcasts a
>> sponsor transaction with a vector of 0123...cdef:0
>>
>> 4. Miners can include that sponsor transaction in any block that has a
>> transaction with an input of 0123...cdef:0. Otherwise the sponsor
>> transaction is consensus invalid.
>>
>> (Note: alternatively, sponsor vectors could point to either txids OR
>> input outpoints. This complicates the serialization of the vector but
>> seems otherwise fine to me.)
>>
>
> *This seems like a fine suggestion and I think addresses Antoine's issue.*
>
>
> *I think there are likely some cases where you do want TXID and not Output
> (e.g., if you *
>
> *are sponsoring a payment to your locktime'd cold storage wallet (no CPFP)
> from an untrusted third party (no RBF), they can grift you into paying for
> an unrelated payment). This isn't a concern when the root utxo is multisig
> & you are a participant.*
>
> *The serialization to support both, while slightly more complicated, can
> be done in a manner that permits future extensibility as well if there are
> other modes people require.*
>
>
>
>>
>> > If we want to solve the hard cases of pinning, I still think mempool
>> > acceptance of a whole package only on the merits of feerate is the
>> easiest
>> > solution to reason on.
>>
>> I don't think package relay based only on feerate solves RBF transaction
>> pinning (and maybe also doesn't solve ancestor/dependent limit pinning).
>> Though, certainly, package relay has the major advantage over this
>> proposal (IMO) in that it doesn't require any consensus changes.
>> Package relay is also very nice for fixing other protocol rough edges
>> that are needed anyway.
>>
>> -Dave
>>
>
> *I think it's important to keep in mind this is not a rival to package
> relay; I think you also want package relay in addition to this, as they
> solve different but related problems.*
>
>
> *Where you might be able to simplify package relay with sponsors is by
> doing a sponsor-only package relay, which is always limited to 2
> transactions, 1 sponsor, 1 sponsoree. This would not have some of the
> challenges with arbitrary-package package-relay, and would (at least from a
> ux perspective) allow users to successfully get parents with insufficient
> fee into the mempool.*
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200922/56a6fd2c/attachment-0001.html>

2023-06-07 18:26:30

- reply

📅 Original date posted:2020-08-24
📝 Original message:Hi all,
Thanks for the helpful discussion.
My primary motivation in starting this thread was to establish what the
expectations are for new feature deployment (particularly whether the
protocol version should continue to be bumped or not), and I think I have
that answer -- different from what I proposed when I started this thread,
but not in a way that I think meaningfully hinders future work. So I'm
happy to leave it at that and withdraw my suggestion.
Cheers,
Suhas
On Sun, Aug 23, 2020 at 1:51 PM Eric Voskuil via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> > On Aug 21, 2020, at 15:16, Matt Corallo <lf-lists at mattcorallo.com>
> wrote:
> >
> > Hmm, could that not be accomplished by simply building this into new
> messages? eg, send "betterprotocol", if you see a verack and no
> "betterprotocol" from your peer, send "worseprotocol" before you send a
> "verack".
> >
> > Matt
> >
> >> On 8/21/20 5:17 PM, Jeremy wrote:
> >> As for an example of where you'd want multi-round, you could imagine a
> scenario where you have a feature A which gets bugfixed by the introduction
> of feature B, and you don't want to expose that you support A unless you
> first negotiate B. Or if you can negotiate B you should never expose A, but
> for old nodes you'll still do it if B is unknown to them.
>
> This seems to imply a security benefit (I can’t discern any other
> rationale for this complexity). It should be clear that this is no more
> than trivially weak obfuscation and not worth complicating the protocol to
> achieve.
>
> >> An example of this would be (were it not already out without a feature
> negotiation existing) WTXID/TXID relay.
> >> The SYNC primitve simply codifies what order messages should be in and
> when you're done for a phase of negotiation offering something. It can be
> done without, but then you have to be more careful to broadcast in the
> correct order and it's not clear when/if you should wait for more time
> before responding.
> >> On Fri, Aug 21, 2020 at 2:08 PM Jeremy <jlrubin at mit.edu <mailto:
> jlrubin at mit.edu>> wrote:
> >> Actually we already have service bits (which are sadly limited)
> which allow negotiation of non bilateral feature
> >> support, so this would supercede that.
> >> --
> >> @JeremyRubin <https://twitter.com/JeremyRubin><
> https://twitter.com/JeremyRubin>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200824/895e4c0a/attachment-0001.html>

2023-06-07 18:26:23

- reply

📅 Original date posted:2020-08-14
📝 Original message:Hi,
Back in February I posted a proposal for WTXID-based transaction relay[1]
(now known as BIP 339), which included a proposal for feature negotiation
to take place prior to the VERACK message being received by each side. In
my email to this list, I had asked for feedback as to whether that proposal
was problematic, and didn't receive any responses.
Since then, the implementation of BIP 339 has been merged into Bitcoin
Core, though it has not yet been released.
In thinking about the mechanism used there, I thought it would be helpful
to codify in a BIP the idea that Bitcoin network clients should ignore
unknown messages received before a VERACK. A draft of my proposal is
available here[2].
I presume that software upgrading past protocol version 70016 was already
planning to either implement BIP 339, or ignore the wtxidrelay message
proposed in BIP 339 (if not, then this would create network split concerns
in the future -- so I hope that someone would speak up if this were a
problem). When we propose future protocol upgrades that would benefit from
feature negotiation at the time of connection, I think it would be nice to
be able to use the same method as proposed in BIP 339, without even needing
to bump the protocol version. So having an understanding that this is the
standard of how other network clients operate would be helpful.
If, on the other hand, this is problematic for some reason, I look forward
to hearing that as well, so that we can be careful about how we deploy
future p2p changes to avoid disruption.
Thanks,
Suhas Daftuar
[1]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-February/017648.html
[2]
https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200814/6d0b3b0f/attachment.html>

2023-06-07 18:26:23

- reply

📅 Original date posted:2020-08-17
📝 Original message:Hi Eric,
Thanks for your response. If I understand correctly, you're suggesting
that in the future we do the same as what was done in BIP 339, of
accompanying new messages (which are optional) with a protocol version
bump, so that network clients are never reading unknown messages from a
peer (and can be free to disconnect a peer for sending an unknown message)?
I think that works fine, so if indeed there will be software that will
expect things to operate this way then I can withdraw the suggestion I've
made in this thread. However I wanted to clarify that this is what you
suggest, because there is another downside to this approach (beyond the
sequential nature of sequence numbers that you mention) -- if a software
implementation misses a proposed new protocol upgrade, and thus fails to
parse (and ignore) some proposed new message, the result can be a network
split down the road as incompatible clients get slowly upgraded over time.
I think this coordination cost is something to be concerned about -- for
instance, the lack of response to my wtxid-relay proposal made me wonder if
other software would be implementing something to account for the new
message that proposal introduces (for clients with version >= 70016). It's
reasonable for people to be busy and miss things like this, and I think
it's worth considering whether there's a safer way for us to deploy changes.
That said, I don't think this coordination cost is unbearable, so as long
as we have a process for making p2p protocol improvements I'm not too
worried about what mechanism we use. So if this concern over coordination
of changes doesn't sway you, I think we can continue to just bump protocol
version at the same time as deploying new messages, as we have been doing,
and hope that we don't run into problems down the road.
If I have misunderstood how you think we should be making future protocol
changes, please let me know.
Thanks,
Suhas
On Sun, Aug 16, 2020 at 3:06 PM Eric Voskuil <eric at voskuil.org> wrote:
> A requirement to ignore unknown (invalid) messages is not only a protocol
> breaking change but poor protocol design. The purpose of version
> negotiation is to determine the set of valid messages. Changes to version
> negotiation itself are very problematic.
>
> The only limitation presented by versioning is that the system is
> sequential. As such, clients that do not wish to implement (or operators
> who do not wish to enable) them are faced with a problem when wanting to
> support later features. This is resolvable by making such features optional
> at the new protocol level. This allows each client to limit its
> communication to the negotiated protocol, and allows ignoring of known but
> unsupported/disabled features.
>
> Sorry I missed your earlier post. Been distracted for a while.
>
> e
>
>
> On Aug 14, 2020, at 12:28, Suhas Daftuar via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>
> Hi,
>
> Back in February I posted a proposal for WTXID-based transaction relay[1]
> (now known as BIP 339), which included a proposal for feature negotiation
> to take place prior to the VERACK message being received by each side. In
> my email to this list, I had asked for feedback as to whether that proposal
> was problematic, and didn't receive any responses.
>
> Since then, the implementation of BIP 339 has been merged into Bitcoin
> Core, though it has not yet been released.
>
> In thinking about the mechanism used there, I thought it would be helpful
> to codify in a BIP the idea that Bitcoin network clients should ignore
> unknown messages received before a VERACK. A draft of my proposal is
> available here[2].
>
> I presume that software upgrading past protocol version 70016 was already
> planning to either implement BIP 339, or ignore the wtxidrelay message
> proposed in BIP 339 (if not, then this would create network split concerns
> in the future -- so I hope that someone would speak up if this were a
> problem). When we propose future protocol upgrades that would benefit from
> feature negotiation at the time of connection, I think it would be nice to
> be able to use the same method as proposed in BIP 339, without even needing
> to bump the protocol version. So having an understanding that this is the
> standard of how other network clients operate would be helpful.
>
> If, on the other hand, this is problematic for some reason, I look forward
> to hearing that as well, so that we can be careful about how we deploy
> future p2p changes to avoid disruption.
>
> Thanks,
> Suhas Daftuar
>
> [1]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-February/017648.html
>
> [2]
> https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200817/f449fb1f/attachment.html>

2023-06-07 18:23:05

- reply

📅 Original date posted:2020-02-25
📝 Original message:Hi all,
I've been working on a proposal to add support for relaying transactions
based on their wtxid, rather than just their txid. The current draft is at
https://github.com/sdaftuar/bips/blob/2020-02-wtxid-relay/bip-wtxid-relay.mediawiki,
and for some background I'll paste the motivation section here:
Historically, the INV messages sent on the Bitcoin peer-to-peer network to
> announce transactions refer to transactions by their txid, which is a hash
> of the transaction that does not include the witness (see BIP 141). This
> has been the case even since Segregated Witness (BIP 141/143/144) has been
> adopted by the network.
>
> Not committing to the witness in transaction announcements creates
> inefficiencies: because a transaction's witness can be malleated without
> altering the txid, a node in receipt of a witness transaction that the node
> does not accept will generally still download that same transaction when
> announced by other peers. This is because the alternative -- of not
> downloading a given txid after rejecting a transaction with that txid --
> would allow a third party to interfere with transaction relay by malleating
> a transaction's witness and announcing the resulting invalid transaction to
> nodes, preventing relay of the valid version of the transaction as well.
>
> We can eliminate this concern by using the wtxid in place of the txid when
> announcing and fetching transactions.
>
One point specifically that I'm seeking feedback on is feature negotiation:
for efficiency, I think it makes sense for peers to negotiate at the
beginning of a connection whether they are going to use wtxid- or
txid-based, prior to announcing any transactions. To achieve this, I
propose in the BIP to send a message between receiving a VERSION message
and prior to sending VERACK (to nodes advertising version at least 70016)
to announce support for this new feature; if both sides send it then they
each know to enable it on the link. My thinking is that in general, it'd
be great to use messages sent between VERSION and VERACK to negotiate
features prior to fully initializing a peer connection (it's sort of a
natural way to extend what we might want to send in a VERSION message,
without breaking existing VERSION-message parsers). However, I don't know
whether inserting a message before VERACK would break any assumptions of
other software on the network, or if this is a problematic paradigm for
some reason, so I'd welcome feedback here.
Thanks,
Suhas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200225/c8652b1f/attachment.html>

2023-06-07 18:16:33

- reply

📅 Original date posted:2019-02-25
📝 Original message:Hi,
I'm writing to report a consensus vulnerability affecting Bitcoin Core
versions
0.13.0, 0.13.1, and 0.13.2. These software versions reached end-of-life on
2018-08-01.
The issue surrounds a fundamental design flaw in Bitcoin's Merkle tree
construction. Last year, the vulnerability (CVE-2017-12842) around 64-byte
transactions being used to trick light clients into thinking that a
transaction
was committed in a block was discussed on this mailing list
(
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-June/016091.html
).
There is a related attack resulting from the ambiguity around 64-byte
transactions which could be used to cause a vulnerable full-node
implementation
to fall out of consensus.
The attack on light clients discussed previously was centered on the idea of
claiming the Merkle tree has one more level in it than it actually has, to
prove that a candidate transaction is in the chain by having its hash match
one
side of a 64-byte transaction. The vulnerability I am describing here
involves
going the other direction: find a row of interior nodes in the Merkle tree
that successfully deserialize as transactions, in order to make a block
appear
to be invalid. (This is of a similar character to the attack described by
Sergio Demian Lerner on
https://bitslog.wordpress.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design/
,
in the section titled "An (expensive) attack to partition Bitcoin".)
It has long been recognized that malleating a block's transactions in a way
that produces the same Merkle root could be used to cause a node to fall
out of
consensus, because of the logic in Bitcoin Core to cache the invalidity of
blocks (ie to avoid re-validation of known-invalid ones, which would
otherwise
make the software vulnerable to DoS). Malleation by "going up" the Merkle
tree,
and claiming that some interior row is in fact the set of (64-byte)
transactions in a block, could be used to cause the Bitcoin Core 0.13
branch to
incorrectly mark as invalid a block that in fact has a valid set of
transactions. Moreover, this requires very little work to accomplish -- less
than 22 bits of work in all.
I have attached a writeup that I put together for my own memory and notes
which
goes into more detail (along with a summary of other Merkle tree issues,
including the duplicate transactions issue from CVE-2012-2459 and the SPV
issue); please see sections 3.1 and 4.1 for a discussion. The bug in 0.13
was
introduced as an unintended side-effect of a change I authored
(https://github.com/bitcoin/bitcoin/pull/7225). Once I learned of this
category of Merkle malleation issues, I realized that the change
inadvertently
introduced a vulnerability to this issue that did not previously exist (in
any
prior version of the software, as far as I can tell). A bug fix that
effectively reverted the change (
https://github.com/bitcoin/bitcoin/pull/9765)
was made just before the 0.14 version of Bitcoin Core was released, and no
later versions of the software are affected.
Also, I have scanned the blockchain looking for instances where the first
two
hashes in any row of the Merkle tree would deserialize validly as a 64-byte
transaction, and I have found zero such instances. So in particular there
are
no blocks on Bitcoin's main chain (as of this writing) that could be used to
attack an 0.13 node.
I thought it best to withhold disclosure of this vulnerability before a
mitigation was in place for the related SPV-issue (which I assumed would
become
obvious with this disclosure); once that became public last summer and a
mitigation deployed (by making 64-byte transactions nonstandard), that
concern
was eliminated.
Thanks to Johnson Lau and Greg Maxwell for originally alerting me to this
issue.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190225/a27d8837/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: BitcoinMerkle.pdf
Type: application/pdf
Size: 200880 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190225/a27d8837/attachment-0001.pdf>

2023-06-07 18:09:54

- reply

📅 Original date posted:2018-01-19
📝 Original message:Hi,
I propose backdating the P2SH and Segwit version 0 script rules back to the
genesis block, as a way to simplify these consensus rules. Here's the
abstract from a draft BIP I wrote up to explain this change:
The Pay to Script Hash (P2SH, BIP 16) script rules and the Version 0
Witness Program script rules (BIP 143/141) can be enforced from the genesis
block with only one historical exception. Doing so simplifies consensus
rules and allows protocol implementers to avoid writing and testing code
paths that are no longer relevant.
The full BIP draft can be found here:
https://github.com/sdaftuar/bips/blob/p2sh-v0segwit-from-genesis/bip-sdaftuar-p2sh-v0segwit-from-genesis.mediawiki
And the currently open pull request to Bitcoin Core which implements this
change can be found here: https://github.com/bitcoin/bitcoin/pull/11739
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180119/9e6d1bb7/attachment.html>

2023-06-07 18:08:41

- reply

📅 Original date posted:2017-12-12
📝 Original message:Hi,
First, thanks for resurrecting this, I agree this is worth pursuing.
On Mon, Dec 11, 2017 at 4:04 PM, Gregory Maxwell via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Nbits _never_ needs to be sent even with other consensus rules because
> its more or less necessarily a strict function of the prior headers.
> This still holds in every clone of Bitcoin I'm aware of; sending it
> with the first header in a group probably makes sense so it can be
> checked independently.
>
I think it would be nice, though, to not require the consensus-correct
calculation of nBits in order to process p2p messages. For instance, I
think there's a use for nBits at the p2p layer for calculating the work on
a chain, which can be used as an anti-DoS measure, even without verifying
that the difficulty adjustments are following the consensus rules.
Moreover I think it's a bit messy if the p2p layer depends on intricate
consensus rules in order to reconstruct a message -- either we'd need to
interact with existing consensus logic in a potentially new way, or we'd
reimplement the same logic in the p2p layer, neither of which is very
desirable imo.
But I think we should be able to get nearly all the benefit just by
including nBits in any messages where the value is ambiguous; ie we include
it with the first header in a message, and whenever it changes from the
previous header's nBits.
I would rather not change the serialization of existing messages,
> nodes are going to have to support speaking both messages for a long
> time, and I think we already want a different protocol flow for
> headers fetching in any case.
>
I agree with this. Specifically the way I envisioned this working is that
we could introduce a new 'cmpctheaders'/'getcmpcthdrs' message pair for
syncing using this new message type, while leaving the existing
'headers'/'getheaders' messages unchanged. So when communicating with
upgraded peers, we'd never use 'getheaders' messages, and we'd only use
'headers' messages for potentially announcing new blocks.
Of course, we'll have to support the existing protocol for a long time.
But one downside I've discovered from deploying BIP 130, which introduced
block announcements via headers messages, is that overloading a 'headers'
message to be either a block announcement or a response to a 'getheaders'
message results in p2p-handling logic which is more complicated than it
needs to be. So splitting off the headers chain-sync functionality to a
new message pair seems like a nice side-effect benefit, in the long run.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20171212/4baa06b9/attachment.html>

2023-06-07 18:00:55

- reply

📅 Original date posted:2017-05-08
📝 Original message:Hi,
I've moved the bitcoin-dev list to bcc:, as this question is better suited
to forums dedicated to Bitcoin Core implementation specifics, rather than
the general bitcoin development list.
Please feel free in the future to ask questions like this on the
bitcoin-core-dev mailing list (https://lists.linuxfoundation
.org/mailman/listinfo/bitcoin-core-dev) or on the #bitcoin-core-dev
freenode IRC channel.
The work limit (that was put in place in https://github.com/bitcoin/
bitcoin/pull/6654, when the concept of "dirty" entries was introduced) was
removed in https://github.com/bitcoin/bitcoin/pull/7594, in preparation for
ancestor-feerate-mining. So those comments should have been cleaned up to
match the new code.
Please feel free to file an issue or open a PR to update those comments at
https://github.com/bitcoin/bitcoin.
Thanks,
Suhas
On Mon, May 8, 2017 at 5:38 AM, DJ Bitcoin via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi Guys,
>
> I have a question about the use of txmempool. find attached the code in
> txmempool.h
>
>
> ======================================================
> /* Adding transactions from a disconnected block can be very time
> consuming,
> * because we don't have a way to limit the number of in-mempool
> descendants.
> * To bound CPU processing, we limit the amount of work we're willing to do
> * to properly update the descendant information for a tx being added from
> * a disconnected block. If we would exceed the limit, then we instead
> mark
> * the entry as "dirty", and set the feerate for sorting purposes to be
> equal
> * the feerate of the transaction without any descendants. */
>
> class CTxMemPoolEntry
> {
> private:
> // ...
> // Information about descendants of this transaction that are in the
> // mempool; if we remove this transaction we must remove all of these
> // descendants as well. if nCountWithDescendants is 0, treat this entry
> as
> // dirty, and nSizeWithDescendants and nModFeesWithDescendants will not
> be
> // correct.
>
> int64_t nCountWithDescendants; //!< number of descendant transactions
> // ...
> ======================================================
>
>
> Now, the only place where nCountWithDescendants is modified is the
> following (txmempool.cpp):
>
>
> ======================================================
> void CTxMemPoolEntry::UpdateDescendantState(int64_t modifySize, CAmount
> modifyFee, int64_t modifyCount)
> {
> nSizeWithDescendants += modifySize;
> assert(int64_t(nSizeWithDescendants) > 0);
> nModFeesWithDescendants += modifyFee;
> nCountWithDescendants += modifyCount;
> assert(int64_t(nCountWithDescendants) > 0);
> }
> ======================================================
>
>
> Therefore, nCountWithDescendants is never zero.
> Am i missing something? Where is this concept of "dirty" defined?
>
> Thanks a lot,
> DJ
>
>
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170508/dfb5c9f0/attachment.html>

2023-06-07 18:00:38

- reply

📅 Original date posted:2017-05-22
📝 Original message:I also do not support the BIP 148 UASF, and I'd like to add to the points
that Greg has already raised in this thread.
BIP 148 would introduce a new consensus rule that softforks out non-segwit
signalling blocks in some time period. I reject this consensus rule as
both arbitrary and needlessly disruptive. Bitcoin's primary purpose is to
reach consensus on the state of a shared ledger, and even though I think
the Bitcoin network ought to adopt segwit, I don't think that concern
trumps the goal of not splitting the network.
Many BIP 148 advocates seem to start with the assumption that segwit
already has a lot of support, and suggest that BIP 148 does as well.
However I don't think it's fair or correct to separate the activation
proposal for segwit from the rest of the segwit proposal. The deployment
parameters for segwit are consensus-critical; assuming that some other
deployment has consensus because it would result in the rest of the segwit
proposal activating is an unjustified leap.
Even if there were no feasible alternate segwit deployment method available
to us, I would hesitate to recommend that the network adopt a potentially
consensus-splitting approach, even though I firmly believe that the ideas
behind segwit are fundamentally good ones. But fortunately that is not the
situation we are in; we have substantially less disruptive methods
available to us to activate it, even if the current BIP 9 deployment were
to fail -- such as another BIP 9 deployment in the future, or perhaps a BIP
149 deployment.
If we do pursue a "user-activated" deployment of segwit, I'd recommend that
we do so in a more careful way than BIP 148 or 149 currently suggest, which
as I understand would otherwise make very few changes to the current
implementation. However, due to the BIP 9 activation assumption, the
Bitcoin Core 0.13.1 - 0.14.0 segwit implementation largely lumps together
the idea that miners would both enforce the rules and mine segwit blocks.
However, we can separate these concerns, as we started to do in the Bitcoin
Core 0.14.1 release, where mining segwit blocks is not required in order to
generally mine or signal for segwit in the software. And we can go further
still: without too much work, we could make further improvements to
accommodate miners who, for whatever reason, don't want to upgrade their
systems, such as by improving block relay from pre-segwit peers [1], or
optimizing transaction selection for miners who are willing to enforce the
segwit rules but haven't upgraded their systems to mine segwit blocks [2].
If we would seek to activate a soft-fork with less clear miner signaling
(such as BIP 149), then I think such improvements are warranted to minimize
network disruption. In general, we should not seek to censor hashpower on
the network unless we have a very important reason for doing so. While the
issues here are nuanced, if I were to evaluate the BIP 148 soft-fork
proposal on the spectrum of "censorship attack on Bitcoin" to "benign
protocol upgrade", BIP 148 strikes me as closer to the former than the
latter. There is simply no need here to orphan these non-signalling blocks
that could otherwise be used to secure the network.
To go further: I think BIP 148 is ill-conceived even for achieving its own
presumed goals -- the motivation for adding a consensus rule that applies
to the version bits on blocks is surely for the effect such bits have on
older software, such as Bitcoin Core releases 0.13.1 and later. Yet in
trying to bring those implementations along as segwit-enforcing software,
BIP 148 would risk forking from such clients in the short term! If one
really cared about maintaining consensus with older, segwit-enabled
software, it would make far more sense to seek segwit activation in a way
that didn't fork from them (such as BIP 149, or a new BIP 9 deployment
after this one times out). And if one doesn't care about such consensus,
then it'd be far simpler to just set (e.g.) August 1 as the flag day
activation of segwit, and not play these contortionist games with block
version bits, which carry no useful or intrinsic meaning. Either of these
two approaches should have the advantage of reduced fork risk, compared
with BIP 148.
Of course, everyone is free to run the software of their choosing. I write
this to both generally convey my opposition to a careless proposal, which I
believe represents a way of thinking that is detrimental to Bitcoin's long
run success, and specifically explain why I oppose inclusion of this
proposal in the Bitcoin Core implementation [3]. The Bitcoin Core project
hasn't been, and shouldn't be, careless in deploying consensus changes.
Instead, I think the Bitcoin Core project ought to stand up for the best
practices that our community has learned about how to deploy such changes
(specifically for minimizing chain-split risk when deploying a soft fork!),
and I think we should all avoid adoption or encouragement of practices that
would depart from the high standards we are capable of achieving.
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017
-March/013811.html
[2] https://github.com/bitcoin/bitcoin/pull/9955
[3] https://github.com/bitcoin/bitcoin/pull/10428#issuecomment-303098925
--Suhas Daftuar
On Fri, Apr 14, 2017 at 3:56 AM, Gregory Maxwell via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> I do not support the BIP148 UASF for some of the same reasons that I
> do support segwit: Bitcoin is valuable in part because it has high
> security and stability, segwit was carefully designed to support and
> amplify that engineering integrity that people can count on now and
> into the future.
>
> I do not feel the the approach proposed in BIP148 really measures up
> to the standard set by segwit itself, or the existing best practices
> in protocol development in this community.
>
> The primary flaw in BIP148 is that by forcing the activation of the
> existing (non-UASF segwit) nodes it almost guarantees at a minor level
> of disruption.
>
> Segwit was carefully engineered so that older unmodified miners could
> continue operating _completely_ without interruption after segwit
> activates.
>
> Older nodes will not include segwit spends, and so their blocks will
> not be invalid even if they do not have segwit support. They can
> upgrade to it on their own schedule. The only risk non-participating
> miners take after segwit activation is that if someone else mines an
> invalid block they would extend it, a risk many miners already
> frequently take with spy-mining.
>
> I do not think it is a horrible proposal: it is better engineered than
> many things that many altcoins do, but just not up to our normal
> standards. I respect the motivations of the authors of BIP 148. If
> your goal is the fastest possible segwit activation then it is very
> useful to exploit the >80% of existing nodes that already support the
> original version of segwit.
>
> But the fastest support should not be our goal, as a community-- there
> is always some reckless altcoin or centralized system that can support
> something faster than we can-- trying to match that would only erode
> our distinguishing value in being well engineered and stable.
>
> "First do no harm." We should use the least disruptive mechanisms
> available, and the BIP148 proposal does not meet that test. To hear
> some people-- non-developers on reddit and such-- a few even see the
> forced orphaning of 148 as a virtue, that it's punitive for
> misbehaving miners. I could not not disagree with that perspective any
> more strongly.
>
> Of course, I do not oppose the general concept of a UASF but
> _generally_ a soft-fork (of any kind) does not need to risk disruption
> of mining, just as segwit's activation does not. UASF are the
> original kind of soft-fork and were the only kind of fork practiced by
> Satoshi. P2SH was activated based on a date, and all prior ones were
> based on times or heights. We introduced miner based activation as
> part of a process of making Bitcoin more stable in the common case
> where the ecosystem is all in harmony. It's kind of weird to see UASF
> portrayed as something new.
>
> It's important the users not be at the mercy of any one part of the
> ecosystem to the extent that we can avoid it-- be it developers,
> exchanges, chat forums, or mining hardware makers. Ultimately the
> rules of Bitcoin work because they're enforced by the users
> collectively-- that is what makes Bitcoin Bitcoin, it's what makes it
> something people can count on: the rules aren't easy to just change.
>
> There have been some other UASF proposals that avoid the forced
> disruption-- by just defining a new witness bit and allowing
> non-upgraded-to-uasf miners and nodes to continue as non-upgraded, I
> think they are vastly superior. They would be slower to deploy, but I
> do not think that is a flaw.
>
> We should have patience. Bitcoin is a system that should last for all
> ages and power mankind for a long time-- ten years from now a couple
> years of dispute will seem like nothing. But the reputation we earn
> for stability and integrity, for being a system of money people can
> count on will mean everything.
>
> If these discussions come up, they'll come up in the form of reminding
> people that Bitcoin isn't easily changed at a whim, even when the
> whims are obviously good, and how that protects it from being managed
> like all the competing systems of money that the world used to use
> were managed. :)
>
> So have patience, don't take short cuts. Segwit is a good improvement
> and we should respect it by knowing that it's good enough to wait for,
> and for however its activated to be done the best way we know how.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170522/8cfe10a6/attachment-0001.html>

2023-06-07 17:57:58

- reply

📅 Original date posted:2017-03-27
📝 Original message:Hi,
There have been two threads recently that have made references to
peer-to-peer implementation details in Bitcoin Core's Segregated Witness
code that I would like to clarify.
In the thread "Issolated Bitcoin Nodes" (
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013765.html),
there was some discussion about how Bitcoin Core's block download logic
behaves after segwit activation. After segwit activation, Bitcoin Core
nodes will not (currently) attempt to download any blocks from non-segwit
peers (nodes that do not set the NODE WITNESS service bit). This is a
bandwidth optimization to prevent a node from downloading a block that may
be invalid only because the sender omitted the witness, requiring
re-download until the block is received with the required witness data.
But to be clear, non-segwit blocks -- that is, blocks without a witness
commitment in the coinbase, and whose transactions are serialized without
witnesses, and whose transactions are not spending segwit outputs which
require a witness -- are evaluated under the same rules as prior,
pre-segwit versions of the software. So such non-segwit blocks that are
valid to older, pre-segwit nodes are also valid to segwit-nodes.
In
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013796.html,
Eric Voskuil wrote:
Given the protocol requirements of the segwit proposal this is not the
> case. A miner running pre-segwit code will produce blocks that no
> segwit node will ever receive.
The phrase "protocol requirements of segwit" is confusing here, because
there are two different layers that need consideration: the consensus
protocol layer and the peer-to-peer protocol layer. But in neither layer
is the behavior of not downloading blocks from non-NODE WITNESS peers a
"requirement". This is an implementation detail in the Bitcoin Core code
that alternate implementations compliant with BIP 144 could implement
differently.
At the consensus layer, non-segwit blocks (described above) that are valid
to older nodes are also valid to segwit nodes. That means that if a miner
was using an older, pre-segwit version of Bitcoin Core to produce blocks
after segwit activates, that blocks they find will be valid to all nodes.
At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only try
to download those blocks if announced by a segwit-enabled peer. But this
is not a protocol requirement; other implementations can remain compatible
even they take different approaches here. (As an example, I could imagine
an implementation that downloaded a new block from any peer, but if the
block has a witness commitment in the coinbase and was received from a
non-segwit peer, then the node would attempt re-download from a segwit
peer. I'm sure many other reasonable block download strategies could be
devised.)
Still, if a miner wants to continue mining post-segwit activation, but
using pre-segwit software, they would need a way to relay their blocks to a
segwit-enabled peer.
There are a few ways to do this that I can think of:
- Use the RPC call "submitblock" on a segwit-enabled node. Calling
"submitblock" on a Bitcoin Core 0.13.1 (0.13.0 in the case of testnet) or
later node works fine as long as the block is valid (whether or not it has
a witness commitment or witness transactions), and once a segwit-enabled
peer has the block it will relay to other segwit peers.
- Explicitly deliver the block to a segwit node over the p2p network, even
if unrequested. Currently Bitcoin Core at least will process unrequested
blocks, and valid blocks that update the tip will then be relayed to other
peers.
- Run a bridge node, which advertises NODE_WITNESS and can serialize blocks
with witness data, which downloads blocks even from non-NODE WITNESS
peers. Anyone can do this to bridge the networks for the benefit of the
whole network (I have personally been running a few nodes that do this, for
several months now), but miners concerned about this issue for their own
blocks could explicitly do this themselves to ensure that their own blocks
propagate to the segwit-enabled network.
- Peer directly with other miners, bypassing the p2p network. Many miners
do this already, using protocols which may already serve to bridge the
network.
So saying that "A miner running pre-segwit code will produce blocks that no
segwit node will ever receive" is not really correct, in my view. If the
whole network were just running Bitcoin Core software releases, and the
miner was not able/willing to deliver their block to a segwit-enabled node
(eg by using the RPC call "submitblock", or one of the other suggestions I
had above), then I would agree with the statement. But given that there
are bridge nodes on the network, and that miners have other options to
relay their block, I think this is not an accurate portrayal of what would
actually happen on the network -- I would expect that non-segwit miners'
blocks would still get relayed post-segwit activation, even if only by the
handful of bridge nodes that I expect are currently running.
All that said, I do think this is an important detail to highlight and that
this behavior should be better documented (I believe it deserves specific
mention in a BIP), as this is an important issue for miners to be aware of.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170327/564a835d/attachment.html>

2023-06-07 17:54:17

- reply

📅 Original date posted:2016-11-15
📝 Original message:Just want to clarify two points:
This change has not yet appeared in any released software (but I assume it
will be in the next release, 0.14.0).
I agree that the performance optimization is not the point of this change;
I can modify the BIP draft to de-emphasize that further (perhaps remove
mention of it entirely).
On Mon, Nov 14, 2016 at 1:47 PM, Eric Voskuil <eric at voskuil.org> wrote:
> NACK
>
> Horrible precedent (hardcoding rule changes based on the assumption that
> large forks indicate a catastrophic failure), extremely poor process
> (already shipped, now the discussion), and not even a material performance
> optimization (the checks are avoidable once activated until a sufficiently
> deep reorg deactivates them).
>
> e
>
> On Nov 14, 2016, at 10:17 AM, Suhas Daftuar via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Hi,
>
> Recently Bitcoin Core merged a simplification to the consensus rules
> surrounding deployment of BIPs 34, 66, and 65 (https://github.com/bitcoin/
> bitcoin/pull/8391), and though the change is a minor one, I thought it
> was worth documenting the rationale in a BIP for posterity.
>
> Here's the abstract:
>
> Prior soft forks (BIP 34, BIP 65, and BIP 66) were activated via miner
> signaling in block version numbers. Now that the chain has long since
> passed the blocks at which those consensus rules have triggered, we can (as
> a simplification and optimization) replace the trigger mechanism by caching
> the block heights at which those consensus rules became enforced.
>
> The full draft can be found here:
>
> https://github.com/sdaftuar/bips/blob/buried-deployments/
> bip-buried-deployments.mediawiki
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20161115/7b2641ce/attachment.html>

2023-06-07 17:54:16

- reply

📅 Original date posted:2016-11-14
📝 Original message:Hi,
Recently Bitcoin Core merged a simplification to the consensus rules
surrounding deployment of BIPs 34, 66, and 65 (
https://github.com/bitcoin/bitcoin/pull/8391), and though the change is a
minor one, I thought it was worth documenting the rationale in a BIP for
posterity.
Here's the abstract:
Prior soft forks (BIP 34, BIP 65, and BIP 66) were activated via miner
signaling in block version numbers. Now that the chain has long since
passed the blocks at which those consensus rules have triggered, we can (as
a simplification and optimization) replace the trigger mechanism by caching
the block heights at which those consensus rules became enforced.
The full draft can be found here:
https://github.com/sdaftuar/bips/blob/buried-deployments/bip-buried-deployments.mediawiki
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20161114/8b1e5b1f/attachment.html>

2023-06-07 17:41:18

- reply

📅 Original date posted:2015-09-24
📝 Original message:On Thu, Sep 24, 2015 at 2:17 PM, Tier Nolan via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Is there actually a requirement for the new message? New nodes could just
> unilaterally switch to sending headers and current nodes would be
> compatible.
>
I don't believe that unilaterally switching to headers announcements would
work for all network participants -- both for users running older Bitcoin
Core versions (anything before 0.10, which I believe all ignore headers
messages) and for non-Bitcoin Core software that participates on the
network (which may ignore headers messages too, I'm not sure what all is
out there).
Even for Bitcoin Core versions 0.10 and 0.11, which process headers and use
them to determine what blocks to download, the block fetching logic is not
optimized for new block announcements via headers messages. Part of what
is implemented in the pull request is direct fetching of blocks upon
receiving a headers message; nodes that don't implement block downloading
in response to headers announcements should continue to receive inv's, I
think -- hence this p2p message to opt-in to the new behavior.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150924/dd63b110/attachment.html>

2023-06-07 17:41:18

- reply

📅 Original date posted:2015-09-24
📝 Original message:I considered that as well, but it seemed to me that other software on the
network (say, different wallet implementations) might prefer the option of
being able to bump up their protocol version in the future to pick up some
other change, without having to also opt-in to receiving
headers-announcements for blocks.
In particular, inv-based block announcements aren't going away (even in my
implementation of headers announcements, there are some edge cases where
the code would need to fall back to an inv announcement), so forcing all
software on the network to upgrade to supporting headers announcements,
whether now or in the future, seems too drastic -- I could imagine some
software not being very concerned about optimizing block relay in this
particular way.
On Thu, Sep 24, 2015 at 2:41 PM, Peter Todd <pete at petertodd.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
>
>
> On 24 September 2015 14:37:40 GMT-04:00, Suhas Daftuar via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
> >On Thu, Sep 24, 2015 at 2:17 PM, Tier Nolan via bitcoin-dev <
> >bitcoin-dev at lists.linuxfoundation.org> wrote:
> >
> >>
> >> Is there actually a requirement for the new message? New nodes could
> >just
> >> unilaterally switch to sending headers and current nodes would be
> >> compatible.
> >>
> >
> >I don't believe that unilaterally switching to headers announcements
> >would
> >work for all network participants -- both for users running older
> >Bitcoin
> >Core versions (anything before 0.10, which I believe all ignore headers
> >messages) and for non-Bitcoin Core software that participates on the
> >network (which may ignore headers messages too, I'm not sure what all
> >is
> >out there).
>
> You can enable the behaviour based on advertised p2p network version.
> -----BEGIN PGP SIGNATURE-----
>
> iQE9BAEBCgAnIBxQZXRlciBUb2RkIDxwZXRlQHBldGVydG9kZC5vcmc+BQJWBEO5
> AAoJEMCF8hzn9Lncz4MH/3ztGWdFvMWWcwQsjIRH+eP6PH57WaEru1smmFYOmKrj
> djdiRVdxfChxRqP3adO21RUKKchjl8DNjrFJHPFz75FSM0cDcD0QAGAHilVdnICE
> LEIlTEoiIc0f1z9f/EJHSHPhiUXMnjpl/l7PYJFZV3Lt2Bl30yLsNnrp9qxjR30n
> 3nykZjyRad4JSavdTP6Evd3qaqwGXNUWsdObXNI+WPKlrw6hczlhFDKQ7RC1FPQU
> Rbgb21pavtqLUTwbBZGUisAAc94e2Gama1p3ioUFklbVtLTdw+FtxPgV/0ZS75OR
> V9pCXIbg9VM6QY4+9gYnP635+qCkqAJ4tBsYGmsT8yA=
> =cF4B
> -----END PGP SIGNATURE-----
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150924/dd34cc2a/attachment-0001.html>

2023-06-07 17:41:17

- reply

📅 Original date posted:2015-09-24
📝 Original message:Hi,
I'm proposing the addition of a new, optional p2p message to help improve
the way blocks are announced on the network. The draft BIP is available
here and pasted below:
https://gist.github.com/sdaftuar/465bf008f0a4768c0def
The goal of this p2p message is to facilitate nodes being able to
optionally announce blocks with headers messages rather than with inv's,
which is particularly beneficial since the introduction of headers-first
download in Bitcoin Core 0.10. In particular, this allows for more
efficient propagation of reorgs as it would eliminate a round trip in
network communication.
The implementation of this BIP (which includes code to directly fetch
blocks based on announced headers) is in
https://github.com/bitcoin/bitcoin/pull/6494. For additional background,
please also see https://github.com/bitcoin/bitcoin/issues/5982.
Note that this new p2p message is optional; nodes can feel free to ignore
and continue to use inv messages to announce new blocks.
Thanks to Pieter Wuille for suggesting this idea.
Draft BIP text:
<pre>
BIP: <unassigned>
Title: sendheaders message
Author: Suhas Daftuar <sdaftuar at chaincode.com>
Status: Draft
Type: Standards Track
Created: 2015-05-08
</pre>
==Abstract==
Add a new message, "sendheaders", which indicates that a node prefers to
receive new block announcements via a "headers" message rather than an
"inv".
==Motivation==
Since the introduction of "headers-first" downloading of blocks in 0.10,
blocks will not be processed unless
they are able to connect to a (valid) headers chain. Consequently, block
relay generally works
as follows:
# A node (N) announces the new tip with an "inv" message, containing the
block hash
# A peer (P) responds to the "inv" with a "getheaders" message (to request
headers up to the new tip) and a "getdata" message for the new tip itself
# N responds with a "headers" message (with the header for the new block
along with any preceding headers unknown to P) and a "block" message
containing the new block
However, in the case where a new block is being announced that builds on
the tip, it would be generally more efficient if the node N just announced
the block header for the new block, rather than just the block hash, and
saved the peer from generating and transmitting the getheaders message (and
the required block locator).
In the case of a reorg, where 1 or more blocks are disconnected, nodes
currently just send an "inv" for the new tip. Peers currently are able to
request the new tip immediately, but wait until the headers for the
intermediate blocks are delivered before requesting those blocks. By
announcing headers from the last fork point leading up to the new tip in
the block announcement, peers are able to request all the intermediate
blocks immediately.
==Specification==
# The sendheaders message is defined as an empty message where pchCommand
== "sendheaders"
# Upon receipt of a "sendheaders" message, the node will be permitted, but
not required, to announce new blocks by sending the header of the new block
(along with any other blocks that a node believes a peer might need in
order for the block to connect).
# Feature discovery is enabled by checking protocol version >= 70012
==Backward compatibility==
Older clients remain fully compatible and interoperable after this change.
==Implementation==
https://github.com/bitcoin/bitcoin/pull/6494
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150924/42c7569e/attachment.html>

2023-06-07 15:44:17

- reply

📅 Original date posted:2015-07-30
📝 Original message:On Thu, Jul 30, 2015 at 12:20 PM, Gavin Andresen via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> So we'd get to 2MB blocks in the year 2021. I think that is much too
> conservative, and the most likely effect of being that conservative is that
> the main blockchain becomes a settlement network, affordable only for
> large-value transactions.
>
While that may be true I think that's okay -- if it turns out that this is
too conservative, and we succeed in scaling the system so that it's
non-controversial to increase these limits later via another hard fork, we
would still be free to do so.
It seems to me that everyone who has been arguing recently to increase the
block size limit ought to find this proposal to be a strict improvement
over where we are now, and this proposal seems like it's reasonably likely
to be non-controversial enough to be worth proposing as a hard fork in
Bitcoin Core -- thank you Pieter for putting this together.
For my part, I'd give this a concept ACK, pending hearing further thoughts
from others.
Suhas Daftuar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150730/ddb717c7/attachment.html>