Pieter Wuille [ARCHIVE]

⚠️ This account is not associated with a real person. It's an archival account for mailing list messages.
🗓️ Active from 2011-06-17 to 2023-04-10
💬 Posted a total of 500 messages
📚 Participated in the following categories: Bitcoin-development, BIP 15, ANN, coredump backtrace, RFC, BIP proposal, softfork proposal, BIP, bitcoin-dev, BIP Draft, BIP Proposal, Lightning-dev, Opt-in full-RBF
🔇 Please do not DM this user as it's used for archive purposes only.

Public Key

npub1tjephawh7fdf6358jufuh5eyxwauzrjqa7qn50pglee4tayc2ntqcjtl6r

Profile Code

nprofile1qqs9evsm7htlyk5ag6rewy7t6vjr8w7ppeqwlqf68s50uu647jv9f4sprpmhxue69uhhyetvv9ujumn0wdmksetjv5hxxmmdqyt8wumn8ghj7un9d3shjtnwdaehgu3wvfskueqngt7f9

Publishing to

relay.noswhere.com

relay.nostr.band

Author Public Key

npub1tjephawh7fdf6358jufuh5eyxwauzrjqa7qn50pglee4tayc2ntqcjtl6r

Show more details

Last Notes

2023-09-07 11:52:36

- reply

📅 Original date posted:2023-09-04
🗒️ Summary of this message: Version 0.4.0 of libsecp256k1 has been released, featuring the addition of the ellswift module for ElligatorSwift encoding. See release notes for details.
📝 Original message:
Hello,
Today we'd like to announce the release of version 0.4.0 of libsecp256k1:
https://github.com/bitcoin-core/secp256k1/releases/tag/v0.4.0
The highlight is the addition of the ellswift module, which implements ElligatorSwift encoding. For the full release notes of 0.4.0 (and earlier releases) see:
https://github.com/bitcoin-core/secp256k1/blob/master/CHANGELOG.md
--
Pieter

2023-06-07 18:27:24

- reply

📅 Original date posted:2020-10-20
📝 Original message:On Tuesday, October 20, 2020 3:29 AM, David A. Harding <dave at dtrt.org> wrote:
> I would guess that some of the failures / stuck transactions might now be successes if the backend infrastructure has upgraded to Bitcoin Core > = 0.19.
Yeah, it would be good to re-test them since a ~year has passed since the 0.19.0 release.
Cheers,
--
Pieter

2023-06-07 18:27:23

- reply

📅 Original date posted:2020-10-19
📝 Original message:On Sunday, October 18, 2020 5:49 PM, Rusty Russell <rusty at rustcorp.com.au> wrote:
> Pieter Wuille bitcoin-dev at wuille.net writes:
>
> > Today, no witness v1 receivers exist. So it seems to me the only question
> > is what software/infrastructure exist that supports sending to witness v1,
> > and whether they (and their userbase) are more or less likely to upgrade
> > before receivers appear than those that don't.
> > Clearly if only actively developed software currently supports sending to
> > v1 right now, then the question of forward compatibility is moot, and I'd
> > agree the cleanliness of option 2 is preferable.
>
> If software supports v1 today and doesn't get upgraded, and we persue
> option 1 then a trailing typo can make trouble. Not directly lose money
> (since the tx won't get propagated), but for most systems (e.g. hosted
> wallets) someone has to go in and figure out the error and fix it up.
It depends. As is, they'd be relayed even as sending to future witness versions
or lengths is standard. If option 1 is chosen there may be reasons to add
safeguards using relay policy, though.
> Option 2 means they're likely to fix their systems the first time
> someone tries a v1 send, not the first time someone makes a trailing
> typo (which might be years).
Possibly, but it's also possible that it won't get fixed at all, and instead
receiver software just has to wait a few years longer before being able to start
giving out v1 addresses and have a reasonable chance the sender supports it.
You're right though that protecting old sender software from being protected
against the insertion bug is a good argument in favor of Option 2.
Strictly speaking it also has an issue, as the error detection properties aren't
guaranteed for new-scheme-address + intended-detected-error interpreted as
old-scheme-address (in particular, you can make 4 substitution errors in
a new-scheme address and have it be a valid old-scheme address). This is much
less of an issue than the insertion bug that remains present with Option 1 in
old senders.
> > As for how long: new witness version/length combinations are only rarely needed,
> > and there are 14 length=32 ones left to pick. We'll likely want to use those
> > first anyway, as it's the cheapest option with 128-bit collision resistance.
> > Assuming future constructions have something like BIP341's leaf versioning, new
> > witness version/length combinations are only required for:
> >
> > - Changes to the commitment structure of script execution (e.g. Graftroot,
> > different hash function for Merkle trees, ...)
> >
> > - Upgrades to new signing cryptography (EC curve change, PQC, ...).
> > - Changes to signatures outside of a commitment structure (e.g. new sighash
> > modes for the keypath in BIP341, or cross-input aggregation for them).
> >
> >
> > and in general, not for things like new script opcodes, or even for fairly
> > invasive redesigns of the script language itself.
>
> Hmm, good point. These can all be done with version bumps.
>
> The only use for extra bytes I can see is per-UTXO flags, but even then
> I can't see why you'd need to know them until their spent (in which case
> you stash the flag in the input, not the output).
>
> And fewer bytes seems bad for fungibility, since multisig would be
> dangerous.
>
> But the future keeps surprising me, so I'm still hesitant.
Of course, our thinking here may change significantly over time - still, I expect
it'll be years before something other than 32-byte addresses is desired.
> > TL;DR: what codebases/services/infrastructure exists today that supports
> > sending to witness v1 BIP173 addresses?
>
> OK, time to waste some money!
>
> Can you provide a mainnet v1 address, and I'll try to spam it from as
> many things as I can find. If we're really lucky, you can collect it
> post-fork and donate it to charity. Or a miner can steal it pre-fork :)
Here is a BIP341 witness v1 address, corresponding to just the generator as
inner public key (using TapTweak(pubkey) as tweak, as suggested by the BIP):
bc1pmfr3p9 YOU j00pfxjh WILL 0zmgp99y8zf LOSE tmd3s5pmedqhy MONEY ptwy6lm87hf5ss52r5n8
Cheers,
--
Pieter

2023-06-07 18:27:22

- reply

📅 Original date posted:2020-10-16
📝 Original message:Hi Rusty,
thanks for starting this thread. We definitely should make a decision around
this soon.
On Wednesday, October 14, 2020 6:40 PM, Rusty Russell via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
> > > Here's a summary of each proposal:
> > > Length restrictions (future segwits must be 10, 13, 16, 20, 23, 26, 29,
> > > 32, 36, or 40 bytes)
> > >
> > > 1. Backwards compatible for v1 etc; old code it still works.
> > > 2. Restricts future segwit versions, may require new encoding if we
> > > want a diff length (or waste chainspace if we need to have a padded
> > > version for compat).
> > >
> > > Checksum change based on first byte:
> > >
> > > 1. Backwards incompatible for v1 etc; only succeeds 1 in a billion.
> > > 2. Weakens guarantees against typos in first two data-part letters to
> > > 1 in a billion.[1]
> > >
> If we go for option 2, v1 (generated from bitcoin core) will simply fail
> the first time you try test it. So it will force an upgrade. There
> are fewer places generating addresses than accepting them, so this
> seems the most likely scenario.
>
> OTOH, with option 1, anyone accepting v1 addresses today is going to
> become a liability once v1 addresses start being generated.
Today, no witness v1 receivers exist. So it seems to me the only question
is what software/infrastructure exist that supports sending to witness v1,
and whether they (and their userbase) are more or less likely to upgrade
before receivers appear than those that don't.
Clearly if only actively developed software currently supports sending to
v1 right now, then the question of forward compatibility is moot, and I'd
agree the cleanliness of option 2 is preferable.
Does anyone have an up-to-date overview of where to-future-witness sending
is supported? I know Bitcoin Core does.
> > It took a lot of community effort to get widespread support for bech32
> > addresses. Rather than go through that again, I'd prefer we use the
> > backwards compatible proposal from BIPs PR#945 and, if we want to
> > maximize safety, consensus restrict v1 witness program size, e.g. reject
> > transactions with scriptPubKeys paying v1 witness programs that aren't
> > exactly 32 bytes.
>
> Yes, I too wish we weren't here. :(
>
> Deferring a hard decision is not useful unless we expect things to be
> easier in future, and I only see it getting harder as time passes and
> userbases grow.
Possibly, but in the past I think there has existed a pattern where adoption
of new technology is at least partially based on certain infrastructure
and codebases going out of business and/or being replaced with newer ones,
rather than improvements to existing ones.
If that effect is significant, option 1 may be preferable: it means less
compatibility issues in the short term, and longer term all that may be
required is fixing the spec, and waiting long enough for old/unmaintained code
to be replaced.
As for how long: new witness version/length combinations are only rarely needed,
and there are 14 length=32 ones left to pick. We'll likely want to use those
first anyway, as it's the cheapest option with 128-bit collision resistance.
Assuming future constructions have something like BIP341's leaf versioning, new
witness version/length combinations are only required for:
* Changes to the commitment structure of script execution (e.g. Graftroot,
different hash function for Merkle trees, ...)
* Upgrades to new signing cryptography (EC curve change, PQC, ...).
* Changes to signatures outside of a commitment structure (e.g. new sighash
modes for the keypath in BIP341, or cross-input aggregation for them).
and in general, not for things like new script opcodes, or even for fairly
invasive redesigns of the script language itself.
> The good news it that the change is fairly simple and the reference
> implementations are widely used so change is not actually that hard
> once the decision is made.
Indeed. Whatever observations we had about adoption of base58 -> bech32 may not
apply because the change to a different checksum is fairly trivial compared to
that. Still, presence of production codebases that just don't update at all
may complicate this.
> > Hopefully by the time we want to use segwit v2, most software will have
> > implemented length limits and so we won't need any additional consensus
> > restrictions from then on forward.
>
> If we are prepared to commit to restrictions on future addresses.
>
> We don't know enough to do that, however, so I'm reluctant; I worry that
> a future scheme where we could save (e.g.) 2 bytes will impractical due
> to our encoding restrictions, resulting in unnecessary onchain bloat.
I'm opposed to consensus-invalidating certain length/version combinations, if
that's what you're suggesting, and I don't think there is a need for it.
TL;DR: what codebases/services/infrastructure exists today that supports
sending to witness v1 BIP173 addresses?
Cheers,
--
Pieter

2023-06-07 18:13:32

- reply

📅 Original date posted:2018-07-04
📝 Original message:On Wed, Jul 4, 2018 at 6:19 AM, matejcik <jan.matejek at satoshilabs.com> wrote:
> hello,
>
> we still have some concerns about the BIP as currently proposed - not
> about the format or data contents, but more about strictness and
> security properties. I have raised some in the previous e-mails, but
> they might have been lost in the overall talk about format.
>
> * Choosing from duplicate keys when combining.
> We believe that "choose whichever value it wishes" is not a good
> resolution strategy. We propose to either change this to "in case of
> conflicts, software MUST reject the conflicting PSBTs", or explain in
> more detail why picking at random is a safe choice.
Outlawing conflicting values would imply forcing all Signers to
implement fixed deterministic nonce generation, which I don't think it
very desirable. Otherwise PSBTs that got copied and signed and
combined again may fail. So I think we should see it the other way: we
choose the keys in such a way that picking arbitrarily is safe. If
there really is a future extension for which it would not be the case
that picking arbitrarily is acceptable, more data can be moved to the
keys, and leave the actual resolution strategy to the Finalizer. That
way Combiners can remain dumb and not need script-specific logic in
every interaction.
An alternative would be to have a fixed resolution strategy (for
example, when combining multiple PSBTs, pick the value from the first
one that has a particular key set), but I don't think this adds very
much - if picking the first is fine, picking a arbitrary one should be
fine too.
> * Signing records with unknown keys.
> There's been some talk about this at start, but there should be a clear
> strategy for Signers when unknown fields are encountered. We intend to
> implement the rule: "will not sign an input with any unknown fields
> present".
> Maybe it is worth codifying this behavior in the standard, or maybe
> there should be a way to mark a field as "optional" so that strict
> Signers know they can _safely_ ignore the unknown field.
Can you envision a situation in which this is needed? In every
scenario I can come up with, the worst that can happen is that the
resulting signature is just invalid. For example, if PSBT existed
before segwit, and then was later extended to support it, a pre-segwit
signer would not recognize that BIP143 would need to be used for
segwit inputs, and produce signatures using the old sighashing
algorithm. The result is just an invalid signature.
I believe that what you're trying to accomplish is preventing signing
something you don't understand, but that's an independent issue.
Signers generally will want to inspect the transaction they're
signing, or ask for confirmation w.r.t. fees or payment destinations
involved. The case where unknown fields are present for a reason you'd
want to withhold signing for will generally also just be the situation
where you don't understand the transaction you're signing.
Here is (perhaps far fetched) example of why it may not be desirable
to reject unknown fields when signing. Imagine an extension is defined
which adds pay-to-contract derivation for keys (Q = P + H(Q||C)G);
this would be a field similar to the current BIP32 derivation one, but
instead give a base key P and a contract C. Now say there is a 2-of-2
multisig in which you're one signer, and the other signer is (unknown
to you) using P2C. After the other party Updating, the input would
contain a P2C field which you don't understand - but it also isn't
something you care about or affects you.
I would not be opposed to having fields with an explicit flag bit that
says "Don't sign if you don't understand this", but I expect that that
can also be left for future extensions.
> * Fields with empty keys.
> This might be inferred from the definition, but is probably worth
> spelling out explicitly: If a field definition states that the key data
> is empty, an implementation MUST enforce this and reject PSBTs that
> contain non-empty data.
> We suggest adding something to the effect of:
> "If a key or value data in a field doesn't match the specified format,
> the PSBT is invalid. In particular, if key data is specified as "none"
> but the key contains data beyond the type specifier, implementation MUST
> reject the PSBT."
> (not sure about the languge, this should of course allow processing
> unknown fields)
Completely agree here. Any implementation that understands a
particular field must enforce whatever structure the field is known to
have.
> * "Combiner can detect inconsistencies"
> Added in response to this comment [1], the current wording looks like
> it's describing what the Combiner is _capable of_, as opposed to
> prescribing what the combiner is _allowed to_ do.
> We suggest changing to something like:
> "For every field type that the Combiner understands, it MAY also refuse
> to combine PSBTs that have inconsistencies in that field, or cause a
> conflict when combined."
Agree, just because Combiners are expected to work correctly on
unknown fields doesn't mean they can't enforce extra consistency
checks on known fields.
Cheers,
--
Pieter

2023-06-07 18:13:14

- reply

📅 Original date posted:2018-06-27
📝 Original message:On Wed, Jun 27, 2018, 07:04 matejcik <jan.matejek at satoshilabs.com> wrote:
> hello,
>
> On 26.6.2018 22:30, Pieter Wuille wrote:
> >> (Moreover, as I wrote previously, the Combiner seems like a weirdly
> >> placed role. I still don't see its significance and why is it important
> >> to correctly combine PSBTs by agents that don't understand them. If you
> >> have a usecase in mind, please explain.
> >
> > Forward compatibility with new script types. A transaction may spend
> > inputs from different outputs, with different script types. Perhaps
> > some of these are highly specialized things only implemented by some
> > software (say HTLCs of a particular structure), in non-overlapping
> > ways where no piece of software can handle all scripts involved in a
> > single transaction. If Combiners cannot deal with unknown fields, they
> > won't be able to deal with unknown scripts.
>
> Record-based Combiners *can* deal with unknown fields. Either by
> including both versions, or by including one selected at random. This is
> the same in k-v model.
>
Yes, I wasn't claiming otherwise. This was just a response to your question
why it is important that Combiners can process unknown fields. It is not an
argument in favor of one model or the other.
> combining must be done independently by Combiner implementations for
> > each script type involved. As this is easily avoided by adding a
> > slight bit of structure (parts of the fields that need to be unique -
> > "keys"), this seems the preferable option.
>
> IIUC, you're proposing a "semi-smart Combiner" that understands and
> processes some fields but not others? That doesn't seem to change
> things. Either the "dumb" combiner throws data away before the "smart"
> one sees it, or it needs to include all of it anyway.
>
No, I'm exactly arguing against smartness in the Combiner. It should always
be possible to implement a Combiner without any script specific logic.
> No, a Combiner can pick any of the values in case different PSBTs have
> > different values for the same key. That's the point: by having a
> > key-value structure the choice of fields can be made such that
> > Combiners don't need to care about the contents. Finalizers do need to
> > understand the contents, but they only operate once at the end.
> > Combiners may be involved in any PSBT passing from one entity to
> > another.
>
> Yes. Combiners don't need to care about the contents.
> So why is it important that a Combiner properly de-duplicates the case
> where keys are the same but values are different? This is a job that,
> AFAICT so far, can be safely left to someone along the chain who
> understands that particular record.
>
That's because PSBTs can be copied, signed, and combined back together. A
Combiner which does not deduplicate (at all) would end up having every
original record present N times, one for each copy, a possibly large blowup.
For all fields I can think of right now, that type of deduplication can be
done through whole-record uniqueness.
The question whether you need whole-record uniqueness or specified-length
uniqueness (=what is offered by a key-value model) is a philosophical one
(as I mentioned before). I have a preference for stronger invariants on the
file format, so that it becomes illegal for a PSBT to contain multiple
signatures for the same key for example, and implementations do not need to
deal with the case where multiple are present.
It seems that you consider the latter PSBT "invalid". But it is well
> formed and doesn't contain duplicate records. A Finalizer, or a
> different Combiner that understands field F, can as well have the rule
> "throw away all but one" for this case.
>
It's not about considering. We're writing a specification. Either it is
made invalid, or not.
In a key-value model you can have dumb combiners that must pick one of the
keys in case of duplication, and remove the necessity of dealing with
duplication from all other implementations (which I consider to be a good
thing). In a record-based model you cannot guarantee deduplication of
records that permit repetition per type, because a dumb combiner cannot
understand what part is supposed to be unique. As a result, a record-based
model forces you to let all implementations deal with e.g. multiple partial
signatures for a single key. This is a minor issue, but in my view shows
how records are a less than perfect match for the problem at hand.
To repeat and restate my central question:
> Why is it important, that an agent which doesn't understand a particular
> field structure, can nevertheless make decisions about its inclusion or
> omission from the result (based on a repeated prefix)?
>
Again, because otherwise you may need a separate Combiner for each type of
script involved. That would be unfortunate, and is very easily avoided.
Actually, I can imagine the opposite: having fields with same "key"
> (identifying data), and wanting to combine their "values" intelligently
> without losing any of the data. Say, two Signers producing separate
> parts of a combined-signature under the same common public key?
>
That can always be avoided by using different identifying information as
key for these fields. In your example, assuming you're talking about some
form of threshold signature scheme, every party has their own "shard" of
the key, which still uniquely identifies the participant. If they have no
data that is unique to the participant, they are clones, and don't need to
interact regardless.
> In case of BIP32 derivation, computing the pubkeys is possibly
> > expensive. A simple signer can choose to just sign with whatever keys
> > are present, but they're not the only way to implement a signer, and
> > even less the only software interacting with this format. Others may
> > want to use a matching approach to find keys that are relevant;
> > without pubkeys in the format, they're forced to perform derivations
> > for all keys present.
>
> I'm going to search for relevant keys by comparing master fingerprint; I
> would expect HWWs generally don't have index based on leaf pubkeys.
> OTOH, Signers with lots of keys probably aren't resource-constrained and
> can do the derivations in case of collisions.
>
Perhaps you want to avoid signing with keys that are already signed with?
If you need to derive all the keys before even knowing what was already
signed with, you've already performed 80% of the work.
> If you take the records model, and then additionally drop the
> > whole-record uniqueness constraint, yes, though that seems pushing it
> > a bit by moving even more guarantees from the file format to
> > application level code.
>
> The "file format" makes no guarantees, because the parsing code and
> application code is the same anyway. You could say I'm proposing to
> separate these concerns ;)
>
Of course a file format can make guarantees. If certain combinations of
data in it do not satsify the specification, the file is illegal, and
implementations do not need to deal with it. Stricter file formats are
easier to deal with, because there are less edge cases to consider.
To your point: proto v2 afaik has no way to declare "whole record
uniqueness", so either you drop that (which I think is unacceptable - see
the copy/sign/combine argument above), or you deal with it in your
application code.
Cheers,
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180627/fecd0346/attachment-0001.html>

2023-06-07 18:13:13

- reply

📅 Original date posted:2018-06-26
📝 Original message:On Tue, Jun 26, 2018 at 8:33 AM, matejcik via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> I'm still going to argue against the key-value model though.
>
> It's true that this is not significant in terms of space. But I'm more
> concerned about human readability, i.e., confusing future implementers.
> At this point, the key-value model is there "for historical reasons",
> except these aren't valid even before finalizing the format. The
> original rationale for using key-values seems to be gone (no key-based
> lookups are necessary). As for combining and deduplication, whether key
> data is present or not is now purely a stand-in for a "repeatable" flag.
> We could just as easily say, e.g., that the high bit of "type" specifies
> whether this record can be repeated.
I understand this is a philosophical point, but to me it's the
opposite. The file conveys "the script is X", "the signature for key X
is Y", "the derivation for key X is Y" - all extra metadata added to
inputs of the form "the X is Y". In a typed record model, you still
have Xes, but they are restricted to a single number (the record
type). In cases where that is insufficient, your solution is adding a
repeatable flag to switch from "the first byte needs to be unique" to
"the entire record needs to be unique". Why just those two? It seems
much more natural to have a length that directly tells you how many of
the first bytes need to be unique (which brings you back to the
key-value model).
Since the redundant script hashes were removed by making the scripts
per-input, I think the most compelling reason (size advantages) for a
record based model is gone.
> (Moreover, as I wrote previously, the Combiner seems like a weirdly
> placed role. I still don't see its significance and why is it important
> to correctly combine PSBTs by agents that don't understand them. If you
> have a usecase in mind, please explain.
Forward compatibility with new script types. A transaction may spend
inputs from different outputs, with different script types. Perhaps
some of these are highly specialized things only implemented by some
software (say HTLCs of a particular structure), in non-overlapping
ways where no piece of software can handle all scripts involved in a
single transaction. If Combiners cannot deal with unknown fields, they
won't be able to deal with unknown scripts. That would mean that
combining must be done independently by Combiner implementations for
each script type involved. As this is easily avoided by adding a
slight bit of structure (parts of the fields that need to be unique -
"keys"), this seems the preferable option.
> ISTM a Combiner could just as well combine based on whole-record
> uniqueness, and leave the duplicate detection to the Finalizer. In case
> the incoming PSBTs have incompatible unique fields, the Combiner would
> have to fail anyway, so the Finalizer might as well do it. Perhaps it
> would be good to leave out the Combiner role entirely?)
No, a Combiner can pick any of the values in case different PSBTs have
different values for the same key. That's the point: by having a
key-value structure the choice of fields can be made such that
Combiners don't need to care about the contents. Finalizers do need to
understand the contents, but they only operate once at the end.
Combiners may be involved in any PSBT passing from one entity to
another.
> There's two remaining types where key data is used: BIP32 derivations
> and partial signatures. In case of BIP32 derivation, the key data is
> redundant ( pubkey = derive(value) ), so I'd argue we should leave that
> out and save space. In case of partial signatures, it's simple enough to
> make the pubkey part of the value.
In case of BIP32 derivation, computing the pubkeys is possibly
expensive. A simple signer can choose to just sign with whatever keys
are present, but they're not the only way to implement a signer, and
even less the only software interacting with this format. Others may
want to use a matching approach to find keys that are relevant;
without pubkeys in the format, they're forced to perform derivations
for all keys present.
And yes, it's simple enough to make the key part of the value
everywhere, but in that case it becomes legal for a PSBT to contain
multiple signatures for a key, for example, and all software needs to
deal with that possibility. With a stronger uniqueness constraint,
only Combiners need to deal with repetitions.
> Thing is: BIP174 *is basically protobuf* (v2) as it stands. If I'm
> succesful in convincing you to switch to a record set model, it's going
> to be "protobuf with different varint".
If you take the records model, and then additionally drop the
whole-record uniqueness constraint, yes, though that seems pushing it
a bit by moving even more guarantees from the file format to
application level code. I'd like to hear opinions of other people who
have worked on implementations about changing this.
Cheers,
--
Pieter

2023-06-07 18:13:09

- reply

📅 Original date posted:2018-06-22
📝 Original message:On Thu, Jun 21, 2018 at 12:56 PM, Peter D. Gray via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> I have personally implemented this spec on an embedded micro, as
> the signer and finalizer roles, and written multiple parsers for
> it as well. There is nothing wrong with it, and it perfectly meets
> my needs as a hardware wallet.
This is awesome to hear. We need to hear from people who have comments
or issues they encounter while implementing, but also cases where
things are fine as is.
> So, there is a good proposal already spec'ed and implemented by
> multiple parties. Andrew has been very patiently shepherding the PR
> for over six months already.
>
> PSBT is something we need, and has been missing from the ecosystem
> for a long time. Let's push this out and start talking about future
> versions after we learn from this one.
I understand you find the suggestions being brought up in this thread
to be bikeshedding over details, and I certainly agree that "changing
X will gratuitously cause us more work" is a good reason not to make
breaking changes to minutiae. However, at least abstractly speaking,
it would be highly unfortunate if the fact that someone implemented a
draft specification results in a vested interest against changes which
may materially improve the standard.
In practice, the process surrounding BIPs' production readiness is not
nearly as clear as it could be, and there are plenty of BIPs actually
deployed in production which are still marked as draft. So in reality,
truth is that this thread is "late", and also why I started the
discussion by asking what the state of implementations was. As a
result, the discussion should be "which changes are worth the hassle",
and not "what other ideas can we throw in" - and some of the things
brought up are certainly the latter.
So to get back to the question what changes are worth the hassle - I
believe the per-input derivation paths suggested by matejcik may be
one. As is written right now, I believe BIP174 requires Signers to
pretty much always parse or template match the scripts involved. This
means it is relatively hard to implement a Signer which is compatible
with many types of scripts - including ones that haven't been
considered yet. However, if derivation paths are per-input, a signer
can just produce partial signatures for all keys it has the master
for. As long as the Finalizer understands the script type, this would
mean that Signers will work with any script. My guess is that this
would be especially relevant to devices where the Signer
implementation is hard to change, like when it is implemented in a
hardware signer directly.
What do you think?
Cheers,
--
Pieter

2023-06-07 18:13:06

- reply

📅 Original date posted:2018-06-21
📝 Original message:On Thu, Jun 21, 2018 at 4:29 AM, matejcik <jan.matejek at satoshilabs.com> wrote:
> In the case of everything per-input, the naive Signer can do this:
> 1. (in the global section) pre-serialize the transaction
> 2. (in each input) find and fill out scriptPubKey from the provided UTXO
> 3. (for a given BIP32 path) check if the master fingerprint matches
> mine, if yes, derive secret key, output pubkey, signature
> 4. goto 3 (more keys per input), goto 2 (next input)
>
> Note that this flow works perfectly for multisig; it’s going to be the
> job of a Finalizer to build the final scriptSig, but each input can have
> multiple partial signatures -- and, interestingly, the naive Signer
> doesn’t even need to know about multisig.
Ah, you're thinking of an even simpler signer than I was imagining. I
don't think this works in general, because the hash being signed
depends on the structure of the script. For example, if it is P2SH, it
is the redeemscript that goes into the scriptCode serialization rather
than the scriptPubKey. If it is segwit, BIP143 serialization needs to
be used, etc. It may work if your signing is restricted to just one of
those structures, though.
> A less naive Signer will want to check things, maybe derive a scriptSig
> itself and check if it matches the given hash, etc., but it can do this
> all in place. You go linearly through the signing flow and place a
> couple strategic assertions along the way.
Right - but I think anything but the simplest signer must do this,
just to be able to distinguish between different kinds of signature
hashing.
But you're right, having per-input redeemscript/witnessscript
simplifies things still - instead of needing to look a script hash in
a map, you can just compare it with *the* redeemscript/witnessscript.
> However, if the data is global, as is now, it gets more complicated:
> 1. (in the global section) pre-serialize the transaction, prefill lookup
> tables
> 2. (for a given BIP32 path) check if mine, then derive public key and
> store in a dictionary
> 3. (for each input) find _and parse_ scriptPubKey, extract (PK or)
> script hash
> 4. lookup redeem script based on script-hash; if not found, goto 2; if
> found, parse out public key
> 5. lookup public key in the BIP32 dictionary; if not found, goto 2
> 6. output pubkey, signature
I understand your point now. I hadn't considered the possibility of
just signing with all BIP32 derivation paths given for which the
master matches, instead of extracting pubkeys/pkhs from the script.
That's a major simplification for signers indeed. I do think you need
some conditions before to determine the script structure (see above),
but this is a good point in favour of making the derivation paths
per-input.
> In general, you seem to focus a lot on the role of Combiners, esp.
> simple Combiners. To me, that doesn’t look like a significant role. As I
> envision it, a Combiner really doesn’t need to do anything more
> complicated than merge and deduplicate records, simply based on the
> uniqueness of the whole record.
It's more a side-effect of focusing on forward compatibility. I expect
that we will have transactions with inputs spending different kinds of
outputs, and some signers may not be able to understand all of them.
However, as long as the design goal of having Combiners function
correctly for things they don't understand, everything should be able
to work together fine.
> It’s the Finalizer’s job to reconstruct and validate the result. Also
> ISTM if something messes up the PSBT (such as including multiple
> conflicting fields anywhere), it’s OK to leave it to Finalizer to fail.
Agree.
> An aside to this in particular, I’ve been thinking about the requirement
> to share derivation paths and public keys with the Creator. The spec
> assumes that this will happen; you’re talking about providing full
> xpub+chaincode too. At least, the Creator must prefill BIP32 paths and
> master key fingerprints. Possibly also prefill public keys in the redeem
> scripts?
>
> This might not be an improvement proposal, but a point worth being
> raised and maybe explained in the spec. Perhaps the original Creator
> doesn’t have access to this data, and delegates this to some
> “sub-Creators” - I imagine a coordinator sending a PSBT to signing
> parties, each of which acts as a sub-Creator (fills out derivation paths
> and public keys) and a Signer (forwarding to a HWW). Some of the
> discussion even suggests some sort of generic “key derivation field”
> with arbitrary contents - fingerprint + bip32 path? xpub + chain code?
> derivation points? encrypted xprv?
That makes sense - I think we've already touched this when discussing
the requirement for UTXOs to be added. Perhaps those aren't added by
the Creator, but by some index server. The same could be true for the
scripts or derivations paths.
And indeed, most of the information in the derivation paths is
effectively opaque to the Creator - it's just some data given out by
the Signer about its keys that gets passed back to it so it can
identify the key. There is benefit in keeping it in a fixed structure
(like xpub/chaincode, or fingerprint + derivation indexes), to
guarantee compatibility between multiple signer implementations with
access to the same key.
On Tue, Jun 19, 2018 at 5:39 PM, Jason Les via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
>>Hex encoding?
>
> I was hoping for some standard here was well and I agree using something
> more compact than hex is important. My understanding is Z85 is better for
> use with JSON than Base64, which is probably a good benefit to have here.
Both Base64 and Z85 can be stored in JSON strings without quoting
(neither uses quotation characters or backslashes), but Z85 is
slightly more compact (Z85 is 5 characters for 4 bytes, Base64 is 4
characters for 3 bytes). Both use non-alphanumeric characters, so I
don't think there is much difference w.r.t. copy-pastability either.
Z85 is far less common though.
On Thu, Jun 21, 2018 at 4:44 AM, Tomas Susanka via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
>> I think here it makes sense because there can actually only be (up to)
>> one redeemscript and (up to) one witnessscript. So if we made those
>> per-input and per-output, it may simplify signers as they don't need a
>> table lookup to find the correct one. That would also mean we can drop
>> their hashes, even if we keep a key-value model.
> Yes, indeed. Just to clarify: in the first sentence you mean "per
> output", right? There can actually only be (up to) one redeemscript and
> (up to) one witnessscript *per output*.
Up to one per output, and up to one per input - indeed.
On Thu, Jun 21, 2018 at 7:32 AM, Tomas Susanka via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
>>> A question to consider is,
>>> will there be more per-output data? If yes, it might make sense to have
>>> an output section.
>> I think it is unlikely that there would be anymore per-output data.
>
> Hmm, upon further reflection, maybe it's not even worth including *any*
> per-output data, aside from what the original transaction contains.
>
> The output redeem script is either:
> - unknown, because we have received only an address from the receiver
> - or it is known, because it is ours and in that case it doesn’t make
> sense to include it in PSBT
>
> We got stuck on the idea of the Creator providing future (output)
> redeem/witness scripts. But that seems to be a minority use case and can
> be solved efficiently via the same channels that coordinate the PSBT
> creation. Sorry to change opinions so quickly on this one.
Perhaps you're missing the reason for having output scripts? It is so
that signers that wish to known the amounts transferred can be told
which outputs of the to-be transaction are change, and thus shouldn't
be counted towards the balance. By providing the scripts and
derivation paths in a PSBT, the Creator can prove to the Signer that
certain outputs do not actually move funds to some other entity.
Based on the points before, my preference is having everything
per-input and per-output except the transaction (with empty
scriptSig/witness) itself, and having exactly one set/map per input
and output (which may include a "finalized scriptSig/witness field"
for finalized inputs). The overhead of having at least one separator
byte for every input and output in the transaction is at most a few
percent compared to the data in the transaction itself. If size is
really an issue (but I think we've already established that small size
gains aren't worth much extra complexity), we could also serialize the
transaction without scriptSigs/witnesses (which are at least one byte
each, and guaranteed to be empty).
I'm unsure about typed record vs. key-value model. If we'd go with a
per-input script approach, the key would just be a single byte ("the
redeemscript" and "the witnessscript"), so the advantage of being able
to drop the script hashes applies equally to both models. After that,
it seems the only difference seems to be that a well-defined prefix of
the records is enforced to be unique as opposed to the entire record
being enforced to be unique. I don't think there is much difference in
complexity, as Combiners and Signers still need to enforce some kind
of uniqueness even in a typed records model.
Cheers,
--
Pieter

2023-06-07 18:13:04

- reply

📅 Original date posted:2018-06-19
📝 Original message:On Tue, Jun 19, 2018 at 7:20 AM, matejcik via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
Thanks for your comments so far. I'm very happy to see people dig into
the details, and consider alternative approaches.
> 1) Why isn't the global type 0x03 (BIP-32 path) per-input? How do we
> know, which BIP-32 path goes to which input? The only idea that comes to
> my mind is that we should match the input's scriptPubKey's pubkey to
> this 0x03's key (the public key).
> If our understanding is correct, the BIP-32 path is global to save space
> in case two inputs share the same BIP-32 path? How often does that
> happen? And in case it does, doesn't it mean an address reuse which is
> discouraged?
Yes, the reason is address reuse. It may be discouraged, but it still
happens in practice (and unfortunately it's very hard to prevent
people from sending to the same address twice).
It's certainly possible to make them per-input (and even per-output as
suggested below), but I don't think it gains you much. At least when a
signer supports any kind of multisig, it needs to match up public keys
with derivation paths. If several can be provided, looking them up
from a global table or a per-input table shouldn't fundamentally
change anything.
However, perhaps it makes sense to get rid of the global section
entirely, and make the whole format a transaction plus per-input and
per-output extra fields. This would result in duplication in case of
key reuse, but perhaps that's worth the complexity reduction.
> 2) The global items 0x01 (redeem script) and 0x02 (witness script) are
> somewhat confusing. Let's consider only the redeem script (0x01) to make
> it simple. The value description says: "A redeem script that will be
> needed to sign a Pay-To-Script-Hash input or is spent to by an output.".
> Does this mean that the record includes both input's redeem script
> (because we need to sign it), but also a redeem script for the output
> (to verify we are sending to a correct P2SH)? To mix those two seems
> really confusing.
>
> Yet again, adding a new output section would make this more readable. We
> would include the input’s redeem script in the input section and the
> output’s redeem script again in the output section, because they’ll most
> likely differ anyway.
I think here it makes sense because there can actually only be (up to)
one redeemscript and (up to) one witnessscript. So if we made those
per-input and per-output, it may simplify signers as they don't need a
table lookup to find the correct one. That would also mean we can drop
their hashes, even if we keep a key-value model.
> 3) The sighash type 0x03 says the sighash is only a recommendation. That
> seems rather ambiguous. If the field is specified shouldn't it be binding?
Perhaps, yes.
> 4) Is it a good idea to skip records which types we are unaware of? We
> can't come up with a reasonable example, but intuitively this seems as a
> potential security issue. We think we should consider introducing a
> flag, which would define if the record is "optional". In case the signer
> encounters a record it doesn't recognize and such flag is not set, it
> aborts the procedure. If we assume the set model we could change the
> structure to <type><optional flag><length>{data}. We are not keen on
> this, but we wanted to include this idea to see what you think.
Originally there was at least this intuition for why it shouldn't be
necessary: the resulting signature for an input is either valid or
invalid. Adding information to a PSBT (which is what signers do)
either helps with that or not. The worst case is that they simply
don't have enough information to produce a signature together. But an
ignored unknown field being present should never result in signing the
wrong thing (they can always see the transaction being signed), or
failing to sign if signing was possible in the first place. Another
way of looking at it, the operation of a signer is driven by queries:
it looks at the scriptPubKey of the output being spent, sees it is
P2SH, looks for the redeemscript, sees it is P2WSH, looks for the
witnessscript, sees it is multisig, looks for other signers'
signatures, finds enough for the threshold, and proceeds to sign and
create a full transaction. If at any point one of those things is
missing or not comprehensible to the signer, he simply fails and
doesn't modify the PSBT.
However, if the sighash request type becomes mandatory, perhaps this
is not the case anymore, as misinterpreting something like this could
indeed result in an incorrect signature.
If we go down this route, if a field is marked as mandatory, can you
still act as a combiner for it? Future extensions should always
maintain the invariant that a simple combiner which just merges all
the fields and deduplicates should always be correct, I think. So such
a mandatory field should only apply to signers?
> In general, the standard is trying to be very space-conservative,
> however is that really necessary? We would argue for clarity and ease of
> use over space constraints. We think more straightforward approach is
> desired, although more space demanding. What are the arguments to make
> this as small as possible? If we understand correctly, this format is
> not intended for blockchain nor for persistent storage, so size doesn’t
> matter nearly as much.
I wouldn't say it's trying very hard to be space-conservative. The
design train of thought started from "what information does a signer
need", and found a signer would need information on the transaction to
sign, and on scripts to descend into, information on keys to derive,
and information on signatures provided by other participants. Given
that some of this information was global (at least the transaction),
and some of this information was per-input (at least the signatures),
separate scopes were needed for those. Once you have a global scope,
and you envision a signer which looks up scripts and keys in a map of
known ones (like the signing code in Bitcoin Core), there is basically
no downside to make the keys and scripts global - while giving space
savings for free to deduplication.
However, perhaps that's not the right way to think about things, and
the result is simpler if we only keep the transaction itself global,
and everything else per-input (and per-output).
I think there are good reasons to not be gratuitously large (I expect
that at least while experimenting, people will copy-paste these things
a lot and page-long copy pastes become unwieldy quickly), but maybe
not at the cost of structural complexity.
On Tue, Jun 19, 2018 at 7:22 AM, matejcik via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> hello,
> this is our second e-mail with replies to Pieter's suggestions.
>
> On 16.6.2018 01:34, pieter.wuille at gmail.com (Pieter Wuille) wrote:
>> * Key-value map model or set model.
> Just to note, we should probably use varint for the <type> field - this
> allows us, e.g., to create “namespaces” for future extensions by using
> one byte as namespace identifier and one as field identifier.
Agree, but this doesn't actually need to be specified right now. As
the key's (and or value's) interpretation (including the type) is
completely unspecified, an extension can just start using 2-byte keys
(as long as the first byte of those 2 isn't used by another field
already).
>> One exception is the "transaction" record, which needs to be unique.
>> That can either be done by adding an exception ("there can only be one
>> transaction record"), or by encoding it separately outside the normal
>> records (that may also be useful to make it clear that it is always
>> required).
>
> This seems to be the case for some fields already - i.e., an input field
> must have exactly one of Non-witness UTXO or Witness Output. So “adding
> an exception” is probably just a matter of language?
Hmm, I wouldn't say so. Perhaps the transaction's inputs and outputs
are chosen by one entity, and then sent to another entity which has
access to the UTXOs or previous transactions. So while the UTXOs must
be present before signing, I wouldn't say the file format itself must
enforce that the UTXOs are present.
However, perhaps we do want to enforce at-most one UTXO per input. If
there are more potential extensions like this, perhaps a key-value
model is better, as it's much easier to enforce no duplicate keys than
it is to add field-specific logic to combiners (especially for
extensions they don't know about yet).
> We’d also like to note that the “number of inputs” field should be
> mandatory - and as such, possibly also a candidate for outside-record field.
If we go with the "not put signatures/witnesses inside the transaction
until all of them are finalized" suggestion, perhaps the number of
inputs field can be dropped. There would be always one exactly for
each input (but some may have the "final script/witness" field and
others won't).
>> * Derivation from xpub or fingerprint
>>
>> For BIP32 derivation paths, the spec currently only encodes the 32-bit
>> fingerprint of the parent or master xpub. When the Signer only has a
>> single xprv from which everything is derived, this is obviously
>> sufficient. When there are many xprv, or when they're not available
>> indexed by fingerprint, this may be less convenient for the signer.
>> Furthermore, it violates the "PSBT contains all information necessary
>> for signing, excluding private keys" idea - at least if we don't treat
>> the chaincode as part of the private key.
>>
>> For that reason I would suggest that the derivation paths include the
>> full public key and chaincode of the parent or master things are
>> derived from. This does mean that the Creator needs to know the full
>> xpub which things are derived from, rather than just its fingerprint.
>
>
> We don’t understand the rationale for this idea. Do you see a scenario
> where an index on master fingerprint is not available but index by xpubs
> is? In our envisioned use cases at least, indexing private keys by xpubs
> (as opposed to deriving from a BIP32 path) makes no sense.
Let me elaborate.
Right now, the BIP32 fields are of the form <master
fingerprint><childidx><childidx><childidx>...
Instead, I suggest fields of the form <master pubkey><master
chaincode><childidx><childidx><childidx>...
The fingerprint in this case is identical to the first 32 bit of the
Hash160 of <master pubkey>, so certainly no information is lost by
making this change.
This may be advantageous for three reasons:
* It permits signers to have ~thousands of master keys (at which point
32-bit fingerprints would start having reasonable chance for
collisions, meaning multiple derivation attempts would be needed to
figure out which one to use).
* It permits signers to index their master keys by whatever they like
(for example, SHA256 rather than Hash160 or prefix thereof).
* It permits signers who don't store a chaincode at all, and just
protect a single private key.
Cheers,
--
Pieter

2023-06-07 18:09:38

- reply

📅 Original date posted:2018-01-09
📝 Original message:On Jan 9, 2018 13:41, "Mark Friedenbach via bitcoin-dev" <
bitcoin-dev at lists.linuxfoundation.org> wrote:
The use of the alt stack is a hack for segwit script version 0 which has
the clean stack rule. Anticipated future improvements here are to switch to
a witness script version, and a new segwit output version which supports
native MAST to save an additional 40 or so witness bytes. Either approach
would allow getting rid of the alt stack hack. They are not part of the
proposal now because it is better to do things incrementally, and because
we anticipate usage of MAST to better inform these less generic changes.
If the goal is to introduce a native MAST output type later, what is gained
by first having the tailcall semantics?
As far as I can see, this proposal does not have any benefits over Johnson
Lau's MAST idea [1]:
* It is more compact, already giving us the space savings a native MAST
version of the tail call semantics would bring.
* It does not need to work around limitations of push size limits or
cleanstack rules.
* The implementation (of code I've seen) is easier to reason about, as it's
just another case in VerifyScript (which you'd need for a native MAST
output later too) without introducing jumps or loops inside EvalScript.
* It can express the same, as even though the MBV opcode supports proving
multiple elements simultaneously, I don't see a way to use that in the tail
call. Every scenario that consists of some logic before deciding what the
tail call is going to be can be rewritten to have that logic inside each of
the branches, I believe.
* It does not interfere with static analysis (see further).
* Tail call semantics may be easier to extend in the future to enable
semantics that are not compactly representable in either proposal right
now, by allowing a single top-level script may invoke multiple subscripts,
or recursion. However, those sound even riskier and harder to analyse to
me, and I don't think there is sufficient evidence they're needed.
Native MAST outputs would need a new witness script version, which your
current proposal does indeed not need. However, I believe a new script
version will be desirable for other reasons regardless (returning invalid
opcodes to the pool of NOPs available for softforks, for example).
I will make a strong assertion: static analyzing the number of opcodes and
sigops gets us absolutely nothing. It is cargo cult safety engineering. No
need to perpetuate it when it is now in the way.
I'm not sure I agree here. While I'd like to see the separate execution
limits go away, removing them entirely and complicating future ability to
introduce unified costing towards weight of execution cost seems the wrong
way to go.
My reasoning is this: perhaps you can currently make an argument that the
current weight limit is sufficient in preventing overly expensive block
validation costs, due to a minimal ratio between script sizes and their
execution cost. But such an argument needs to rely on assumptions about
sighash caching and low per-opcode CPU time, which may change in the
future. In my view, tail call semantics gratuitously remove or complicate
the ability to reason about the executed code.
One suggestion to reduce the impact of this is limiting the per-script
execution to something proportional to the script size. However, I don't
think that addresses all potential concerns about static analysis (for
example, it doesn't easily let you prove all possible execution paths to a
participant in a smart contract).
Another idea that has been suggested on this list is to mark pushes of
potentially executable code on the stack/witness explicitly. This would
retain all ability to analyse, while still leaving the flexibility of
future extensions to tail call execution. If tail call semantics are
adopted, I strongly prefer an approach like that to blindly throwing out
all limits and analysis.
[1] https://github.com/jl2012/bips/blob/mast/bip-mast.mediawiki
Cheers,
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180109/6c862414/attachment.html>

2023-06-07 17:58:13

- reply

📅 Original date posted:2017-03-28
📝 Original message:On Tue, Mar 28, 2017 at 12:56 PM, Paul Iverson via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> So I think Core can't decide on hard forks like this. It must be left up to
> the users. I think only choice is for Core to add a run-time option to allow
> node operators to increase block size limit, so that this very controversial
> decision is not coming from Core. It must come from the community.
Bitcoin Core's (nor any other software's) maintainers can already not
decide on a hard fork, and I keep being confused by the focus on Core
in this topic. Even if a hard forking change (or lack thereof) was
included into a new release, it is still up to the community to choose
to run the new software. Bitcoin Core has very intentionally no
auto-update feature, as the choice for what network rules to implement
must come from node operators, not developers. Ask yourself this: if a
new Bitcoin Core release would include a new rule that blacklists
<random famous person>'s coins. What do you think would happen? I hope
that people would refuse to update, and choose to run different full
node software.
Core is not special. It is one of many pieces of software that
implement today's Bitcoin consensus rules. If a hardfork is to take
place in a way that does not result in two currencies, it must be
clear that the entire ecosystem will adopt it. Bitcoin Core will not
merge any consensus changes that do not clearly satisfy that
criterion.
--
Pieter

2023-06-07 17:54:54

- reply

📅 Original date posted:2016-12-10
📝 Original message:On Sat, Dec 10, 2016 at 4:23 AM, Daniele Pinna via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> We have models for estimating the probability that a block is orphaned
> given average network bandwidth and block size.
>
> The question is, do we have objective measures of these two quantities?
> Couldn't we target an orphan_rate < max_rate?
>
Models can predict orphan rate given block size and network/hashrate
topology, but you can't control the topology (and things like FIBRE hide
the effect of block size on this as well). The result is that if you're
purely optimizing for minimal orphan rate, you can end up with a single
(conglomerate of) pools producing all the blocks. Such a setup has no
propagation delay at all, and as a result can always achieve 0 orphans.
Cheers,
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20161210/edca6d6b/attachment.html>

2023-06-07 17:50:20

- reply

📅 Original date posted:2016-05-09
📝 Original message:On 05/03/2016 12:13 AM, lf-lists at mattcorallo.com (Matt Corallo) wrote:
> Hi all,
>
> The following is a BIP-formatted design spec for compact block relay
> designed to limit on wire bytes during block relay. You can find the
> latest version of this document at
> https://github.com/TheBlueMatt/bips/blob/master/bip-TODO.mediawiki.
Hi Matt,
thank you for working on this!
> ===New data structures===
> Several new data structures are added to the P2P network to relay
> compact blocks: PrefilledTransaction, HeaderAndShortIDs,
> BlockTransactionsRequest, and BlockTransactions. Additionally, we
> introduce a new variable-length integer encoding for use in these data
> structures.
>
> For the purposes of this section, CompactSize refers to the
> variable-length integer encoding used across the existing P2P protocol
> to encode array lengths, among other things, in 1, 3, 5 or 9 bytes.
This is a not, but I think it's a bit strange to have two separate
variable length integers in the same specification. I understand is one
is already the default for variable-length integers currently, and there
are reasons to use the other one for efficiency reasons in some places,
but perhaps we should aim to get everything using the latter?
> ====New VarInt====
> Variable-length integers: bytes are a MSB base-128 encoding of the number.
> The high bit in each byte signifies whether another digit follows. To make
> sure the encoding is one-to-one, one is subtracted from all but the last
> digit.
Maybe it's worth mentioning that it is based on ASN.1 BER's compressed
integer format (see
https://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf
section 8.1.3.5), though with a small modification to make every integer
have a single unique encoding.
> ====HeaderAndShortIDs====
> A HeaderAndShortIDs structure is used to relay a block header, the short
> transactions IDs used for matching already-available transactions, and a
> select few transactions which we expect a peer may be missing.
>
> |shortids||List of uint64_ts||8*shortids_length bytes||Little
> Endian||The short transaction IDs calculated from the transactions which
> were not provided explicitly in prefilledtxn
I tried to derive what length of short ids is actually necessary (some
write-up is on
https://gist.github.com/sipa/b2eb2e486156b5509ac711edd16153ed but it's
incomplete).
For any reasonable numbers I can come up with (in a very wide range),
the number of bits needed is very well approximated by:
log2(#receiver_mempool_txn * #block_txn_not_in_receiver_mempool /
acceptable_per_block_failure_rate)
For example, with 20000 mempool transactions, 2500 transactions in a
block, 95% hitrate, and a chance of 1 in 10000 blocks to fail to
reconstruct, needed_bits = log2(20000 * 2500 * (1 - 0.95) / 0.0001) =
34.54, or 5 byte txids would suffice.
Note that 1 in 10000 failures may sound like a lot, but this is for each
individual connection, and since every transmission uses separately
salted identifiers, occasional failures should not affect global
propagation. Given that transmission failures due to timeouts, network
connectivity, ... already occur much more frequently than once every few
gigabytes (what 10000 blocks corresponds to), that's probably already
more than enough.
In short: I believe 5 or 6 byte txids should be enough, but perhaps it
makes sense to allow the sender to choose (so he can weigh trying
multiple nonces against increasing the short txid length).
> ====Short transaction IDs====
> Short transaction IDs are used to represent a transaction without
> sending a full 256-bit hash. They are calculated by:
> # single-SHA256 hashing the block header with the nonce appended (in
> little-endian)
> # XORing each 8-byte chunk of the double-SHA256 transaction hash with
> each corresponding 8-byte chunk of the hash from the previous step
> # Adding each of the XORed 8-byte chunks together (in little-endian)
> iteratively to find the short transaction ID
An alternative would be using SipHash-1-3 (a form of SipHash with
reduced iteration counts; the default is SipHash-2-4). SipHash was
designed as a Message Authentication Code, where the security
requirements are much stronger than in our case (in particular, we don't
care about observers being able to finding the key, as the key is just
public knowledge here). One of the designers of SipHash has commented
that SipHash-1-3 for collision resistance in hash tables may be enough:
https://github.com/rust-lang/rust/issues/29754#issuecomment-156073946
Using SipHash-1-3 on modern hardware would take ~32 CPU cycles per txid.
> ===Implementation Notes===
There are a few more heuristics that MAY be used to improve performance:
* Receivers should treat short txids in blocks that match multiple
mempool transactions as non-matches, and request the transactions. This
significantly reduces the failure to reconstruct.
* When constructing a compact block to send, the sender can verify it
against its own mempool to check for collisions, and if so, choose to
either try another nonce, or increase the short txid length.
Cheers,
--
Pieter

2023-06-07 17:34:18

- reply

📅 Original date posted:2015-08-10
📝 Original message:On Aug 7, 2015 11:19 PM, "Sergio Demian Lerner via bitcoin-dev" <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> b. Reduce the block rate to a half (average 5 minute blocks)
>
> Suppose this is a one time hard fork. There no drastic technical problems
with any of them: "SPV" mining and the relay network has shown that block
propagation is not an issue for such as small change. Mining centralization
won't radically change for a 2x adjustment.
I don't understand this. All problems that result from propagation delay
are literally doubled by doing so. Centralization pressure results from the
ratio between propagation time and interblock time. Efficient propagation
algorithms like the relay network make this presumably grow sublinear with
larger blocks, but changing the interblock time affects it exactly
proportionally.
All problems that result from propagation delay are literally doubled by
doing this. Doubling the block size has a smaller effect. You may argue
that these centralization effects are small, but reducing the interblock
time has a stronger effect on them than the block size.
Also, you seem to consider SPV mining a good thing? It requires trust
between miners that know eachother, and fundamentally breaks the security
assumption of SPV clients... and if the propagation/interblock ratio was
lower, SPV mining would have less effect. I'd say it is exactly a result of
the centralization pressure we're trying to avoid.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150810/f50a4a75/attachment-0001.html>

2023-06-07 17:32:36

- reply

📅 Original date posted:2015-08-06
📝 Original message:On Aug 6, 2015 9:42 PM, "Gavin Andresen via bitcoin-dev" <
bitcoin-dev at lists.linuxfoundation.org> wrote:
2. The "market minimum fee" should be determined by the market. It should
not be up to us to decide "when is a good time."
I partially agree. The community should decide what risks it is willing to
take, and set limits accordingly. Let the market decide how that space is
best used.
>
>>
>> Would you agree that blocksize increase proposals should have such a
>> criterion/test?
>
>
> Although I've been very clear with my criterion, no, I don't think all
blocksize increase proposals should have to justify "why this size" or "why
this rate of increase." Part of my frustration with this whole debate is
we're talking about a sanity-check upper-limit; as long as it doesn't open
up some terrible new DoS possibility I don't think it really matters much
what the exact number is.
It is only a DoS protection limit if you want to rely on trusting miners. I
prefer a system where I don't have to do that.
But I agree the numbers don't matter much, for a different reason: the
market will fill up whatever space is available, and we'll have the same
discussion when the new limit doesn't seem enough anymore.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150806/7e4c45e8/attachment.html>

2023-06-07 17:32:33

- reply

📅 Original date posted:2015-08-06
📝 Original message:On Thu, Aug 6, 2015 at 4:21 PM, Gavin Andresen <gavinandresen at gmail.com>
wrote:
> On Thu, Aug 6, 2015 at 10:06 AM, Pieter Wuille <pieter.wuille at gmail.com>
> wrote:
>
>> But you seem to consider that a bad thing. Maybe saying that you're
>> claiming that this equals Bitcoin failing is an exaggeration, but you do
>> believe that evolving towards an ecosystem where there is competition for
>> block space is a bad thing, right?
>>
>
> No, competition for block space is good.
>
So if we would have 8 MB blocks, and there is a sudden influx of users (or
settlement systems, who serve much more users) who want to pay high fees
(let's say 20 transactions per second) making the block chain inaccessible
for low fee transactions, and unreliable for medium fee transactions (for
any value of low, medium, and high), would you be ok with that? If so, why
is 8 MB good but 1 MB not? To me, they're a small constant factor that does
not fundamentally improve the scale of the system. I dislike the outlook of
"being forever locked at the same scale" while technology evolves, so my
proposal tries to address that part. It intentionally does not try to
improve a small factor, because I don't think it is valuable.
What is bad is artificially limiting or centrally controlling the supply of
> that space.
>
It's exactly as centrally limited as the finite supply of BTC - by
consensus. You and I may agree that a finite supply is a good thing, and
may disagree about whether a consensus rule about the block size is a good
idea (and if so, at what level), but it's a choice we make as a community
about the rules of the system we want to use.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150806/d69b8f33/attachment-0001.html>

2023-06-07 17:32:33

- reply

📅 Original date posted:2015-08-06
📝 Original message:On Thu, Aug 6, 2015 at 3:40 PM, Gavin Andresen via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> On Wed, Aug 5, 2015 at 9:26 PM, Jorge Timón <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>> This is a much more reasonable position. I wish this had been starting
>> point of this discussion instead of "the block size limit must be
>> increased as soon as possible or bitcoin will fail".
>>
>
> It REALLY doesn't help the debate when you say patently false statements
> like that.
>
> My first blog post on this issue is here:
> http://gavinandresen.ninja/why-increasing-the-max-block-size-is-urgent
>
> ... and I NEVER say "Bitcoin will fail". I say:
>
> "If the number of transactions waiting gets large enough, the end result
> will be an over-saturated network, busy doing nothing productive. I don’t
> think that is likely– it is more likely people just stop using Bitcoin
> because transaction confirmation becomes increasingly unreliable."
>
But you seem to consider that a bad thing. Maybe saying that you're
claiming that this equals Bitcoin failing is an exaggeration, but you do
believe that evolving towards an ecosystem where there is competition for
block space is a bad thing, right?
I don't agree that "Not everyone is able to use the block chain for every
use case" is the same thing as "People stop using Bitcoin". People are
already not using it for every use case.
Here is what my proposed BIP says: "No hard forking change that relaxes the
block size limit can be guaranteed to provide enough space for every
possible demand - or even any particular demand - unless strong
centralization of the mining ecosystem is expected. Because of that, the
development of a fee market and the evolution towards an ecosystem that is
able to cope with block space competition should be considered healthy."
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150806/aff56c38/attachment.html>

2023-06-07 17:32:27

- reply

📅 Original date posted:2015-08-04
📝 Original message:On Tue, Aug 4, 2015 at 3:12 PM, Gavin Andresen <gavinandresen at gmail.com>
wrote:
> On Tue, Aug 4, 2015 at 7:27 AM, Pieter Wuille via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>> I would say that things already demonstrately got terrible. The mining
>> landscape is very centralized, with apparently a majority depending on
>> agreements to trust each other's announced blocks without validation.
>>
> And that is a problem... why?
>
If miners need to form alliances of trusting each other's blocks without
validation to overcome the inefficiencies of slow block propagation, I
think we have a system that is in direct conflict with the word
"permissionless" that you use later.
> As Bitcoin grows, pieces of the ecosystem will specialize. Satoshi's
> original code did everything: hashing, block assembly, wallet, consensus,
> network. That is changing, and that is OK.
>
Specialization is perfectly fine.
>
> I believe that if the above would have happened overnight, people would
> have cried wolf. But somehow it happened slow enough, and "things kept
> working".
>
> I don't think that this is a good criterion. Bitcoin can "work" with
> gigabyte blocks today, if everyone uses the same few blockchain validation
> services, the same few online wallets, and mining is done by a cartel that
> only allows joining after signing a contract so they can sue you if you
> create an invalid block. Do you think people will then agree that "things
> got demonstratebly worse"?
>
> Don't turn Bitcoin into something uninteresting, please.
>
Why is what you, personally, find interesting relevant?
>
I find it interesting to build a system that has potential to bring about
innovation.
I understand you want to build an extremely decentralized system, where
> everybody participating trusts nothing except the genesis block hash.
>
That is not true, I'm sorry if that is the impression I gave.
I see centralization and scalability as a trade-off, and for better or for
worse, the block chain only offers one trade-off. I want to see technology
built on top that introduces lower levels of trust than typical fully
centralized systems, while offering increased convenience, speed,
reliability, and scale. I just don't think that all of that can happen on
the lowest layer without hurting everything built on top. We need different
trade-offs, and the blockchain is just one, but a very fundamental one.
I think it is more interesting to build a system that works for hundreds of
> millions of people, with no central point of control and the opportunity
> for ANYBODY to participate at any level. Permission-less innovation is what
> I find interesting.
>
That sounds amazing, but do you think that Bitcoin, as it exists today, can
scale to hundreds of millions of users, while retaining any glimpse of
permission-lessness and decentralization? I think we need low-trust
off-chain systems and other innovations to make that happen.
> And I think the current "demonstrably terrible" Bitcoin system is still
> INCREDIBLY interesting.
>
I'm happy for you, then.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150804/1128ad92/attachment.html>

2023-06-07 17:32:23

- reply

📅 Original date posted:2015-08-04
📝 Original message:I would say that things already demonstrately got terrible. The mining
landscape is very centralized, with apparently a majority depending on
agreements to trust each other's announced blocks without validation. Full
node count is at its historically lowest value in years, and outsourcing of
full validation keeps growing.
I believe that if the above would have happened overnight, people would
have cried wolf. But somehow it happened slow enough, and "things kept
working".
I don't think that this is a good criterion. Bitcoin can "work" with
gigabyte blocks today, if everyone uses the same few blockchain validation
services, the same few online wallets, and mining is done by a cartel that
only allows joining after signing a contract so they can sue you if you
create an invalid block. Do you think people will then agree that "things
got demonstratebly worse"?
Don't turn Bitcoin into something uninteresting, please.
--
Pieter
On Aug 4, 2015 1:04 PM, "Hector Chu via bitcoin-dev" <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Mike's position is that he wants the block size limit
> to eventually be removed. That is of course an extreme view. Meanwhile,
> your view that the block size should be artificially constrained below the
> organic growth curve (in a way that will penalize a majority of existing
> and future users) lies at the other extreme. The majority position lies
> somewhere in between (i.e. a one-time increase to 8MB). This is the
> position that ultimately matters.
>
> If the block size is increased to 8MB and things get demonstrably a whole
> lot worse, then you will have a solid leg to stand on. In that case we can
> always do another hard fork later to reduce the block size back to
> something smaller, and henceforth the block size will never be touched
> again.
>
> On 4 August 2015 at 11:35, Jorge Timón <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>> On Fri, Jul 31, 2015 at 4:58 PM, Mike Hearn <hearn at vinumeris.com> wrote:
>> >> How more users or more nodes can bring more miners, or more
>> importantly,
>> >> improve mining decentralization?
>> >
>> >
>> > Because the bigger the ecosystem is the more interest there is in taking
>> > part?
>>
>> As explained by Venzen, this is a non-sequitur.
>>
>> > I mean, I guess I don't know how to answer your question.
>>
>> I don't know the answer either, that's fine. It's the opposite
>> question that I've been insistently repeating and you've been
>> (consciously or not) consistently evading.
>> But that's also fine because I believe you finally answer it a few lines
>> below.
>>
>> > When Bitcoin was
>> > new it had almost no users and almost no miners. Now there are millions
>> of
>> > users and factories producing ASICs just for Bitcoin.
>>
>> The emergence of a btc price enabled the emergence of professional
>> miners, which in turn enabled the emergence of sha256d-specialized
>> hardware production companies.
>> Nothing surprising there.
>> By no means it consitutes an example of how a bigger consensus sizes
>> can cause less mining centralization.
>>
>> > Surely the correlation is obvious?
>>
>> Correlation does not imply causation. I will better leave it at that...
>>
>> >> I'm sorry, but until there's a simulation that I can run with different
>> >> sizes' testchains (for example using #6382) to somehow compare them, I
>> will
>> >> consider any value arbitrary.
>> >
>> >
>> > Gavin did run simulations. 20mb isn't arbitrary, the process behind it
>> was
>> > well documented here:
>> >
>> >
>> http://gavinandresen.ninja/does-more-transactions-necessarily-mean-more-centralized
>> >
>> > I chose 20MB as a reasonable block size to target because 170 gigabytes
>> per
>> > month comfortably fits into the typical 250-300 gigabytes per month data
>> > cap– so you can run a full node from home on a “pretty good” broadband
>> plan.
>> >
>> > Did you think 20mb was picked randomly?
>>
>> No, I think 20 MB was chosen very optimistically, considering 3rd
>> party services rates (not the same service as self-hosting) in the
>> so-called "first world". And then 20 MB goes to 20 GB, again with
>> optimistic and by no means scientific expectations.
>>
>> But where the number comes from it's not really what I'm demaning,
>> what I want is some criterion that can tell you that a given size
>> would be "too centralized" but another one isn't.
>> I haven't read any analysis on why 8GB is a better option than 7GB and
>> 9GB for a given criterion (nor one declaring 20 GB a winner over 19 GB
>> or 21 GB).
>> A simulation test passing 20 GB but not 21 GB would make it far less
>> arbitrary.
>>
>> >> Agreed on the first sentence, I'm just saying that the influence of
>> >> the blocksize in that function is monotonic: with bigger sizes, equal
>> >> or worse mining centralization.
>> >
>> >
>> > I have a hard time agreeing with this because I've seen Bitcoin go from
>> > blocks that were often empty to blocks that are often full, and in this
>> time
>> > the number of miners and hash power on the network has gone up a huge
>> amount
>> > too.
>>
>> I'm of course talking about consensus maximum blocksize, not about
>> actual blocksize.
>> Yes, again, when mining becomes profitable, economic actors tend to
>> appear and get those profits.
>> But don't confuse total hashrate improvements with an "increase in the
>> number of miners" or with mining decentralization.
>>
>> > You can argue that a miner doesn't count if they pool mine. But if a
>> miner
>> > mines on a pool that uses exactly the same software and settings as the
>> > miner would have done anyway, then it makes no difference. Miners can
>> switch
>> > between pools to find one that works the way they like, so whilst less
>> > pooling or more decentralised pools would be nice (e.g.
>> getblocktemplate),
>> > and I've written about how to push it forward before, I still say there
>> are
>> > many more miners than in the past.
>> >
>> > If I had to pick between two changes to improve mining decentralisation:
>> >
>> > 1) Lower block size
>>
>> Finally, I think you finally answered my repetitive question here.
>> If I say "Mike Hearn understands that the consensus block size maximum
>> rule is a tool for limitting mining centralization" I'm not putting
>> words in your mouth, right?
>> I think many users advocating for an increase in the consensus limit
>> don't understand this, which is extremely unfortunate for the debate.
>>
>> > 2) Finishing, documenting, and making the UX really slick for a
>> > getblocktemplate based decentralised mining pool
>> >
>> > then I'd pick (2) in a heartbeat. I think it'd be a lot more effective.
>>
>> Great! Maybe after 2 mining centralization improves so much that we're
>> confortable not only not lowering it but rather increasing it.
>>
>> >> you should be consequently advocating for full removal of the limit
>> rather
>> >> than changes towards bigger arbitrary values.
>> >
>> >
>> > I did toy with that idea a while ago. Of course there can not really be
>> no
>> > limit at all because the code assumes blocks fit into RAM/swap, and
>> nodes
>> > would just end up ignoring blocks they couldn't download in time anyway.
>> > There is obviously a physical limit somewhere.
>>
>> Did the fact that you "understand that the consensus block size
>> maximum rule is a tool for limitting mining centralization" influenced
>> your rejection of that idea at all?
>>
>> > But it is easier to find common ground with others by compromising. Is
>> 8mb
>> > better than no limit? I don't know and I don't care much: I think
>> Bitcoin
>> > adoption is a slow, hard process and we'll be lucky to increase average
>> > usage 8x over the next couple of years. So if 8mb+ is better for others,
>> > that's OK by me.
>>
>> The only way that "not caring much whther we have a consensus limit or
>> not" and "understand that the consensus block size maximum rule is a
>> tool for limitting mining centralization" at the same time is by not
>> caring about mining centralization at all.
>> Is that your position?
>>
>> If you don't care about having a limit but you don't want to limit
>> transaction volume, then ++current_size will ALWAYs be your
>> "compromise position" and no blocksize increase will ever be enough
>> until the limit is completely removed.
>> Is that your position?
>>
>> > Re: exchange profit. You can pick some other useful service provider if
>> you
>> > like. Payment processors or cold storage providers or the TREZOR
>> > manufacturers or whoever.
>>
>> Yes, and I believe the same points stand.
>>
>> > My point is you can't have a tiny high-value-transactions only currency
>> AND
>> > all the useful infrastructure that the Bitcoin community is making.
>> It's a
>> > contradiction. And without the infrastructure bitcoin ceases to be
>> > interesting even to people who are willing to pay huge sums to use it.
>>
>> You keep talking about "high-value-transactions-only" like if
>> non-urgent transaction fees rising from zero to, say, 1 satoshi, would
>> automatically result in that "high-value-transactions-only" Bitcoin.
>> Please, stop talking as if someone was proposing a
>> "high-value-transactions-only" Bitcoin. That may happen but nobody
>> really knows. If it happens it may not be bad thing necessarily (ie
>> bitcoin microtransactions can still happen using trustless payment
>> channels and x is still cheaper than x% for any transacted value
>> higher than 100) but that's really not what we're talking about here
>> so it seems distraction that can only help further polirizing this
>> discussion.
>>
>> What we're talking about here is that hitting the limit would
>> (hopefully) make miners start caring about fees. Enough that they stop
>> being irrational about free transactions. If both things happen,
>> non-urgent transaction fees will likely rise (as said, above zero).
>>
>> You think that would be a catastrophe for adoption and I disagree.
>> But (as Pieter has repeatedly explained) for any size there will be
>> use cases that will be eventually priced out.
>> So when rising this consensus limit, not increasing centralization
>> should be the priority and the potential impact in market fees a much
>> more secondary concern.
>> Do you agree with this?
>>
>> I'm sure there are many intermediate positions between "caring more
>> about mining centralization than market fees when deciding about a
>> consensus rule that limits mining centralization" and "not caring
>> about mining centralization at all".
>> I really don't want to put words in your mouth, but I honestly don't
>> know what your position is.
>> I don't really know how else can I ask the same question: you don't
>> care the consensus maximum blocksize rule being here at all or not
>> (you just said that).
>> Is it because you don't think it limits mining centralization or
>> because you don't care about limiting mining centralization with
>> consensus rules at all?
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150804/2b6fdfa3/attachment.html>

2023-06-07 17:32:10

- reply

📅 Original date posted:2015-08-02
📝 Original message:>>> 2. Starting date: 30 days after 75% miner support, but not before
>>> 2016-01-12 00:00 UTC
>>>
>>> Rationale: A 30-day grace period is given to make sure everyone has
>>> enough time to follow. This is a compromise between 14 day in BIP101
>>> and 1 year in BIP103. I tend to agree with BIP101. Even 1 year is
>>> given, people will just do it on the 364th day if they opt to
>>> procrastinate.
>>
>>
>> Given the time recent softforks have taken to deploy, I think that's
>> too soon.
>
>
> Since I'm using "30 days after 75% miner support", the actual deployment
period will be longer than 30 days. Anyway, if all major exchanges and
merchants agree to upgrade, people are forced to upgrade immediately or
they will follow a worthless chain.
If we don't want it to go fast, why let them? A hardfork is a means for the
community to agree on the rules that different parties have to obey.
>>> 3. The block size at 2016-01-12 will be 1,414,213 bytes, and
>>> multiplied by 1.414213 by every 2^23 seconds (97 days) until exactly
>>> 8MB is reached on 2017-05-11.
>>>
>>> Rationale: Instead of jumping to 8MB, I suggest to increase it
>>> gradually to 8MB in 16 months. 8MB should not be particularly
>>> painful to run even with current equipment (you may see my earlier
>>> post on bitctointalk:
>>> https://bitcointalk.org/index.php?topic=1054482.0 ). 8MB is also
>>>
>>> agreed by Chinese miners, who control >60% of the network.
>>
>>
>> I have considered suggesting a faster ramp-up in the beginning, but I
>> don't think there is indisputable evidence that we can currently deal
>> with significantly larger blocks. I don't think "painful" is the right
>> criterion either; I'm sure my equipment can "handle" 20 MB blocks too,
>> but with a huge impact on network propagation speed, and even more
>> people choosing the outsource their full nodes.
>>
>> Regarding "reasonable", I have a theory. What if we would have had 8
>> MB blocks from the start? My guess is that some more people would have
>> decided to run their high-transaction-rate use cases on chain, that
>> we'd regularly see 4-6 MB blocks, there would be more complaints about
>> low full node counts, maybe 90% instead of 60% of the hash rate would
>> be have SPV mining agreements with each other, we'd somehow have
>> accepted that even worse reality, but people would still be
>> complaining about the measly 25 transactions per second that Bitcoin
>> could handle on-chain, and be demanding a faster rampup to a more
>> "reasonable" 64 MB block size as well.
>
>
> Since the block reward is miners' major income source, no rational miner
would create mega blocks unless the fee could cover the extra orphaning
risk. Blocks were not constantly full until recent months, and many miners
are still keeping the 750kB soft limit. This strongly suggests that we
won't have 4MB blocks now even Satoshi set a 8MB limit.
I disagree. I think "demand" is strongly influenced by the knowledge of
space that looks available. If you look at historic block sizes, you see it
follows a series of block functions, not nice organic growth. My theory is
that this is changed defaults in software, new services appearing suddenly,
and people reacting to it. Demand fills the available space.
Also, SPV mining has nearly zero orphaning risk, only brief chance of loss
of fees as income.
> I don't have the data now but I believe the Satoshi Dice model failed not
primarily due to the 1MB cap, but the raise in BTC/USD rate. Since minting
reward is a fixed value in BTC, the tx fee must also be valued in BTC as it
is primarily for compensating the extra orphaning risk. As the BTC/USD rate
increases, the tx fee measured in USD would also increase, making
micro-payment (measured in USD) unsustainable.
I agree, but how does that matter? I don't think high fees and full blocks
should be the goal, but I think it would be a healthier outcome than what
we have now.
> We might have less full nodes, but it was Satoshi's original plan: "At
first, most users would run network nodes, but as the network grows beyond
a certain point, it would be left more and more to specialists with server
farms of specialized hardware. A server farm would only need to have one
node on the network and the rest of the LAN connects with that one node."
Theoretically, we only require one honest full node to prove wrongdoing on
the blockchain and tell every SPV nodes to blacklist the invalid chain.
Theoretically, we also only need one central bank, then? Sorry, if the
outcome is one (or just a few) entities that keep the system in check, I
think it loses any benefit it has over other systems, while still keeping
its costs and disadvantages (confirmation speed, mining infrastructure,
subsidy...).
I know that 8 MB blocks do not immediately mean such a dramatic outcome.
But I do believe that as a community setting the block size based on
observed demand (which you do by saying "8 is a more reasonable size than
2" as argument) is the wrong way. What do you do when your 8 MB starts to
look "full", before your schedule says it can increase?
The block size and its costs - bandwidth, processing, centralization
effects for miners, ... - are the things that should be constrained. Demand
will fill it.
> I think SPV mining exists long before the 1MB block became full, and I
don't think we could stop this trend by artificially suppressing the block
size. Miners should just do it properly, e.g. stop mining until the
grandparent block is verified, which would make sure an invalid fork won't
grow beyond 2 blocks.
That would be a huge reduction in security as a mechanism itself, and even
worse due to needing to trust miners to follow that protocol. Without
proper incentives, miners become a trusted party, and due to needed SPV
mining agreements, potentially a closed group too. That is not an
interesting future.
> If you believe Bitcoin should become a global settlement network, 32MB
would be the very minimum as that is only 75% of current SWIFT traffic.
See my BIP text about advantages of Bitcoin, please. A future where it has
to compete with the existing system - using that system's own strengths -
is not interesting.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150802/d39a8ed3/attachment.html>

2023-06-07 17:32:08

- reply

📅 Original date posted:2015-08-01
📝 Original message:On Fri, Jul 31, 2015 at 10:39 AM, jl2012 via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> There is a summary of the proposals in my previous mail at
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009808.html
> 1. Initiation: BIP34 style voting, with support of 750 out of the last
> 1000 blocks. The "hardfork bit" mechanism might be used:
> http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009576.html
>
This is fine, I think. I believe we shouldn't proceed with a hardfork
without having reasonable expectation that it will be deployed by everyone
in time, while we can only measure miner acceptance. Still, as a
belt-and-suspenders this won't hurt.
> 2. Starting date: 30 days after 75% miner support, but not before
> 2016-01-12 00:00 UTC
>
> Rationale: A 30-day grace period is given to make sure everyone has enough
> time to follow. This is a compromise between 14 day in BIP101 and 1 year in
> BIP103. I tend to agree with BIP101. Even 1 year is given, people will just
> do it on the 364th day if they opt to procrastinate.
>
Given the time recent softforks have taken to deploy, I think that's too
soon.
2016-01-12 00:00 UTC is Monday evening in US and Tuesday morning in China.
> Most pool operators and devs should be back from new year holiday and not
> sleeping. (If the initiation is delayed, we may require that it must be UTC
> Tuesday midnight).
>
That's an interesting thing to take into account.
> 3. The block size at 2016-01-12 will be 1,414,213 bytes, and multiplied by
> 1.414213 by every 2^23 seconds (97 days) until exactly 8MB is reached on
> 2017-05-11.
>
> Rationale: Instead of jumping to 8MB, I suggest to increase it gradually
> to 8MB in 16 months. 8MB should not be particularly painful to run even
> with current equipment (you may see my earlier post on bitctointalk:
> https://bitcointalk.org/index.php?topic=1054482.0). 8MB is also agreed by
> Chinese miners, who control >60% of the network.
>
I have considered suggesting a faster ramp-up in the beginning, but I don't
think there is indisputable evidence that we can currently deal with
significantly larger blocks. I don't think "painful" is the right criterion
either; I'm sure my equipment can "handle" 20 MB blocks too, but with a
huge impact on network propagation speed, and even more people choosing the
outsource their full nodes.
Regarding "reasonable", I have a theory. What if we would have had 8 MB
blocks from the start? My guess is that some more people would have decided
to run their high-transaction-rate use cases on chain, that we'd regularly
see 4-6 MB blocks, there would be more complaints about low full node
counts, maybe 90% instead of 60% of the hash rate would be have SPV mining
agreements with each other, we'd somehow have accepted that even worse
reality, but people would still be complaining about the measly 25
transactions per second that Bitcoin could handle on-chain, and be
demanding a faster rampup to a more "reasonable" 64 MB block size as well.
>
> 4. After 8MB is reached, the block size will be increased by 6.714% every
> 97 days, which is equivalent to exactly octuple (8x) every 8.5 years, or
> double every 2.9 years, or +27.67% per year. Stop growth at 4096MB on
> 2042-11-17.
>
> Rationale: This is a compromise between 17.7% p.a. of BIP103 and 41.4%
> p.a. of BIP101. This will take us almost 8 years from now just to go back
> to the original 32MB size (4 years for BIP101 and 22 years for BIP103)
>
> SSD price is expected to drop by >50%/year in the coming years. In 2020,
> we will only need to pay 2% of current price for SSD. 98% price reduction
> is enough for 40 years of 27.67% growth.
> Source:
> http://wikibon.org/wiki/v/Evolution_of_All-Flash_Array_Architectures
>
I know many technologies have had faster growth, but I believe that global
bandwidth accessibility is the bottleneck, so the growth rate in my
proposal is based on that.
> Global bandwidth is expected to grow by 37%/year until 2021 so 27.67%
> should be safe at least for the coming 10 years.
> Source:
> https://www.telegeography.com/research-services/global-bandwidth-forecast-service/
>
I'd rather be conservative here. My primary purpose is trying to create an
uncontroversial proposal that introduces an expectation of growth with
technology.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150801/78c2ce7a/attachment.html>

2023-06-07 17:32:08

- reply

📅 Original date posted:2015-08-01
📝 Original message:On Sat, Aug 1, 2015 at 10:22 PM, John T. Winslow via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Regarding the block size increase, at least conceptually it seems to me
> there should be an easy solution. If we look at what works well with
> bitcoin, for example the block reward halving and difficulty regimes which
> due to their step function nature both contribute to the stability and
> predictability of the bitcoin universe while still allowing for the
> necessary dynamic adjustments. It seems to me there should be a
> corresponding and equally simple solution for the maximum block size.
>
> I've never quite understood the supposed rationale behind the proposals
> for a new static maximum of 20 MB or 8 MB or 2 MB other than it would be
> trivial to implement. Why not come up with an equally simple, predictable
> dynamic function consistent with what is already proven to work in the
> bitcoin universe that would both preserve the scarcity of transaction
> capacity to encourage a fee market but also allow for more transactions
> when needed.
>
I disagree with the notion of "needed". The blockchain provides utility at
every size, and perhaps more at some sizes than at other sizes, but any
finite size will permit some use cases and not others. This is already the
case and will remain the case. I think the "demand for payments" should be
considered infinite, and some of them will fit on a block chain and pay a
fee for it, and others will need to rely on more efficient, cheaper, but
higher trust systems. You can't use observed usage as a metric for demand
without fixing the cost.
I think available space should grow with technology, to keep the relative
costs to the ecosystem for maintaining a decentralized system constant.
That may or may not lead to a fee market, but I don't think the fee market
is a goal - only a healthy outcome. The goal is an ecosystem that accepts
that the limit to scalability is set by the requirements of a decentralized
system, and its demand - and certainly not perceived demand at potentially
near-zero fee/cost - can't change that.
> For example how about something like once every month at month-end, take
> the 6-month average non-zero transaction fee block size and multiply by 1.5?
>
>
That's trivially gamable by miners, by filling the blocks with their own
transactions.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150801/df61499e/attachment.html>

2023-06-07 15:46:18

- reply

2023-06-07 15:44:14

- reply

📅 Original date posted:2015-07-30
📝 Original message:Hello all,
here is a proposal for long-term scalability I've been working on:
https://gist.github.com/sipa/c65665fc360ca7a176a6
Some things are not included yet, such as a testnet whose size runs ahead
of the main chain, and the inclusion of Gavin's more accurate sigop
checking after the hard fork.
Comments?
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150730/99e5886b/attachment.html>

2023-06-07 15:41:43

- reply

📅 Original date posted:2015-07-05
📝 Original message:I would say yes. Just putting a locktime in transaction may help against
fee sniping, even in transactions that are allowed to be mined at the same
time as some of their dependencies?
On Jul 5, 2015 6:17 PM, "Mark Friedenbach" <mark at friedenbach.org> wrote:
> Can you construct an example? Are there use cases where there is a need
> for an enforced lock time in a transaction with inputs that are not
> confirmed at the time the lock time expires?
> On Jul 5, 2015 8:00 AM, "Tom Harding" <tomh at thinlink.com> wrote:
>
>> BIP 68 uses nSequence to specify relative locktime, but nSequence also
>> continues to condition the transaction-level locktime.
>>
>> This dual effect will prevent a transaction from having an effective
>> nLocktime without also requiring at least one of its inputs to be mined
>> at least one block (or one second) ahead of its parent.
>>
>> The fix is to shift the semantics so that nSequence = MAX_INT - 1
>> specifies 0 relative locktime, rather than 1. This change will also
>> preserve the semantics of transactions that have already been created
>> with the specific nSequence value MAX_INT - 1 (for example all
>> transactions created by the bitcoin core wallet starting in 0.11).
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150705/f360820c/attachment.html>

2023-06-07 15:40:26

- reply

📅 Original date posted:2015-06-26
📝 Original message:Hello all,
here I'm going to try to address a part of the block size debate which has
been troubling me since the beginning: the reason why people seem to want
it.
People say that larger blocks are necessary. In the long term, I agree - in
the sense that systems that do not evolve tend to be replaced by other
systems. This evolution can come in terms of layers on top of Bitcoin's
blockchain, in terms of the technology underlying various aspects of the
blockchain itself, and also in the scale that this technology supports.
I do, however, fundamentally disagree that a fear for a change in economics
should be considered to necessitate larger blocks. If it is, and there is
consensus that we should adapt to it, then there is effectively no limit
going forward. This is similar to how Congress voting to increase the
copyright term retroactively from time to time is really no different from
having an infinite copyright term in the first place. This scares me.
Here is how Gavin summarizes the future without increasing block sizes in
PR 6341:
> 1. Transaction confirmation times for transactions with a given fee will
rise; very-low-fee transactions will fail to get confirmed at all.
> 2. Average transaction fee paid will rise
> 3. People or applications unwilling or unable to pay the rising fees will
stop submitting transactions
> 4. People and businesses will shelve plans to use Bitcoin, stunting
growth and adoption
Is it fair to summarize this as "Some use cases won't fit any more, people
will decide to no longer use the blockchain for these purposes, and the
fees will adapt."?
I think that is already happening, and will happen at any scale. I believe
demand for payments in general is nearly infinite, and only a small portion
of it will eventually fit on a block chain (independent of whether its size
is limited by consensus rules or economic or technological means).
Furthermore, systems that compete with Bitcoin in this space already offer
orders of magnitude more capacity than we can reasonably achieve with any
blockchain technology at this point.
I don't know what subset of use cases Bitcoin will cater to in the long
term. They have already changed - you see way less betting transactions
these days than a few years ago for example - and they will keep changing,
independent of what effective block sizes we end up with. I don't think we
should be afraid of this change or try to stop it.
If you look at graphs of block sizes over time (for example,
http://rusty.ozlabs.org/?p=498), it seems to me that there is very little
"organic" growth, and a lot of sudden changes (which could correspond to
changing defaults in miner software, introduction of popular
sites/services, changes in the economy). I think these can be seen as the
economy changing to full up the available space, and I believe these will
keep happening at any size effectively available.
None of this is a reason why the size can't increase. However, in my
opinion, we should do it because we believe it increases utility and
understand the risks; not because we're afraid of what might happen if we
don't hurry up. And from that point of view, it seems silly to make a huge
increase at once...
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150626/69d0ee83/attachment.html>

2023-06-07 15:38:41

- reply

📅 Original date posted:2015-06-18
📝 Original message:On Thu, Jun 18, 2015 at 3:31 PM, Mike Hearn <mike at plan99.net> wrote:
> OK, let's agree to unpack the two things.
>
> The first issue is how are decisions made in Bitcoin Core? I struggle to
> explain this to others because I don't understand it myself. Is it a vote
> of people with commit access? Is it a 100% agreement of "core developers"
> and if so, who are these people? Is it "whoever reverts the change last"?
> Could I write down in a document a precise description of how decisions are
> made? No, and that's been a fairly frustrating problem for a long time.
>
> But let's leave it to one side for a moment.
>
> Let's focus on the other issue: what happens if the Bitcoin Core
> decision making process goes wrong?
>
Why do you keep talking about Bitcoin Core maintainers? The means for doing
a hard fork is convincing the network to run modified code, whether that is
a new version of Bitcoin Core or a fork of it, or something else entirely.
If I see consensus about a proposed network change, I will be in favor of
implementing it in Bitcoin Core. But we're not at that point. There is no
network change proposed with consensus. There is not even a patch to be
discussed. There are working proposals, and people are talking about them.
This is good.
I think maintainers of particular software should not be, and are not those
who decide the network's rules. People running the code are. Of course
maintainers have a large influence, but so do other people - like you.
> This was a reference to a post by Gregory on Reddit where he said if
Gavin were to do a pull request for the block size change and then merge
it, he would revert it. And I fully believe he would do so!
I believe so too, and I would do the same. Because I believe implementing a
consensus rule change without having very good expectations that the
network will adopt it, is reckless from the point of view of maintainers,
for all reasons I have mentioned before.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150618/d8c40122/attachment.html>

2023-06-07 15:37:24

- reply

📅 Original date posted:2015-06-12
📝 Original message:I'm merely proving the existence of the effect.
On Jun 12, 2015 8:24 PM, "Mike Hearn" <mike at plan99.net> wrote:
> are only connected to each other through a slow 2 Mbit/s link.
>>
>
> That's very slow indeed. For comparison, plain old 3G connections
> routinely cruise around 7-8 Mbit/sec.
>
> So this simulation is assuming a speed dramatically worse than a mobile
> phone can get!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150612/1a3a0e82/attachment.html>

2023-06-07 15:37:23

- reply

📅 Original date posted:2015-06-12
📝 Original message:Hello all,
I've created a simulator for Bitcoin mining which goes a bit further than
the one Gavin used for his blog post a while ago. The main difference is
support for links with different latency and bandwidth, because of the
clustered configuration described below. In addition, it supports different
block sizes, takes fees into account, does difficulty adjustments, and
takes processing and mining delays into account. It also simulates longer
periods of time, and averages the result of many simulations running in
parallel until the variance on the result is low enough.
The code is here: https://github.com/sipa/bitcoin-net-simul
The configuration used in the code right now simulates two groups of miners
(one 80%=25%+25%+30%, one 20%=5%+5%+5%+5%), which are well-connected
internally, but are only connected to each other through a slow 2 Mbit/s
link.
Here are some results.
This shows how the group of smaller miners loses around 8% of their
relative income (if they create larger blocks, their loss percentage goes
up slightly further):
Configuration:
* Miner group 0: 80.000000% hashrate, blocksize 20000000.000000
* Miner group 1: 20.000000% hashrate, blocksize 1000000.000000
* Expected average block size: 16200000.000000
* Average fee per block: 0.250000
* Fee per byte: 0.0000000154
Result:
* Miner group 0: 81.648985% income (factor 1.020612 with hashrate)
* Miner group 1: 18.351015% income (factor 0.917551 with hashrate)
When fees become more important however, and half of a block's income is
due to fees, the effect becomes even stronger (a 15% loss), and the optimal
way to compete for small miners is to create larger blocks as well (smaller
blocks for them result in even less income):
Configuration:
* Miner group 0: 80.000000% hashrate, blocksize 20000000.000000
* Miner group 1: 20.000000% hashrate, blocksize 20000000.000000
* Expected average block size: 20000000.000000
* Average fee per block: 25.000000
* Fee per byte: 0.0000012500
Result:
* Miner group 0: 83.063545% income (factor 1.038294 with hashrate)
* Miner group 1: 16.936455% income (factor 0.846823 with hashrate)
The simulator is not perfect. It doesn't take into account that multiple
blocks/relays can compete for the same bandwidth, or that nodes cannot
process multiple blocks at once.
The numbers used may be unrealistic, and I don't mean this as a prediction
for real-world events. However, it does very clearly show the effects of
larger blocks on centralization pressure of the system. Note that this also
does not make any assumption of destructive behavior on the network - just
simple profit maximalization.
Lastly, the code may be buggy; I only did some small sanity tests with
simple networks.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150612/e8c2f530/attachment.html>

2023-06-07 15:37:23

- reply

📅 Original date posted:2015-06-12
📝 Original message:If there is a benefit in producing larger more-fee blocks if they propagate
slowly, then there is also a benefit in including high-fee transactions
that are unlikely to propagate quickly through optimized relay protocols
(for example: very recent transactions, or out-of-band receoved ones). This
effect is likely an order of magnitude less important still, but the effect
is likely the same.
On Jun 12, 2015 8:31 PM, "Peter Todd" <pete at petertodd.org> wrote:
> On Fri, Jun 12, 2015 at 01:21:46PM -0400, Gavin Andresen wrote:
> > Nice work, Pieter. You're right that my simulation assumed bandwidth for
> > 'block' messages isn't the bottleneck.
> >
> > But doesn't Matt's fast relay network (and the work I believe we're both
> > planning on doing in the near future to further optimize block
> propagation)
> > make both of our simulations irrelevant in the long-run?
>
> Then simulate first the relay network assuming 100% of txs use it, and
> secondly, assuming 100%-x use it.
>
> For instance, is it in miners' advantage in some cases to sabotage the
> relay network? The analyse say yes, so lets simulate that. Equally even
> the relay network isn't instant.
>
> --
> 'peter'[:-1]@petertodd.org
> 0000000000000000127ab1d576dc851f374424f1269c4700ccaba2c42d97e778
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150612/ba1f8c93/attachment.html>

2023-06-07 15:35:08

- reply

📅 Original date posted:2015-05-13
📝 Original message:Normalized transaction ids are only effectively non-malleable when all
inputs they refer to are also non-malleable (or you can have malleability
in 2nd level dependencies), so I do not believe it makes sense to allow
mixed usage of the txids at all. They do not provide the actual benefit of
guaranteed non-malleability before it becomes disallowed to use the old
mechanism. That, together with the +- resource doubling needed for the UTXO
set (as earlier mentioned) and the fact that an alternative which is only a
softfork are available, makes this a bad idea IMHO.
Unsure to what extent this has been presented on the mailinglist, but the
softfork idea is this:
* Transactions get 2 txids, one used to reference them (computed as
before), and one used in an (extended) sighash.
* The txins keep using the normal txid, so not structural changes to
Bitcoin.
* The ntxid is computed by replacing the scriptSigs in inputs by the empty
string, and by replacing the txids in txins by their corresponding ntxids.
* A new checksig operator is softforked in, which uses the ntxids in its
sighashes rather than the full txid.
* To support efficiently computing ntxids, every tx in the utxo set
(currently around 6M) stores the ntxid, but only supports lookup bu txid
still.
This does result in a system where a changed dependency indeed invalidates
the spending transaction, but the fix is trivial and can be done without
access to the private key.
On May 13, 2015 5:50 AM, "Christian Decker" <decker.christian at gmail.com>
wrote:
> Hi All,
>
> I'd like to propose a BIP to normalize transaction IDs in order to address
> transaction malleability and facilitate higher level protocols.
>
> The normalized transaction ID is an alias used in parallel to the current
> (legacy) transaction IDs to address outputs in transactions. It is
> calculated by removing (zeroing) the scriptSig before computing the hash,
> which ensures that only data whose integrity is also guaranteed by the
> signatures influences the hash. Thus if anything causes the normalized ID
> to change it automatically invalidates the signature. When validating a
> client supporting this BIP would use both the normalized tx ID as well as
> the legacy tx ID when validating transactions.
>
> The detailed writeup can be found here:
> https://github.com/cdecker/bips/blob/normalized-txid/bip-00nn.mediawiki.
>
> @gmaxwell: I'd like to request a BIP number, unless there is something
> really wrong with the proposal.
>
> In addition to being a simple alternative that solves transaction
> malleability it also hugely simplifies higher level protocols. We can now
> use template transactions upon which sequences of transactions can be built
> before signing them.
>
> I hesitated quite a while to propose it since it does require a hardfork
> (old clients would not find the prevTx identified by the normalized
> transaction ID and deem the spending transaction invalid), but it seems
> that hardforks are no longer the dreaded boogeyman nobody talks about.
> I left out the details of how the hardfork is to be done, as it does not
> really matter and we may have a good mechanism to apply a bunch of
> hardforks concurrently in the future.
>
> I'm sure it'll take time to implement and upgrade, but I think it would be
> a nice addition to the functionality and would solve a long standing
> problem :-)
>
> Please let me know what you think, the proposal is definitely not set in
> stone at this point and I'm sure we can improve it further.
>
> Regards,
> Christian
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150513/bb0fcadf/attachment.html>

2023-06-07 15:33:05

- reply

📅 Original date posted:2015-05-07
📝 Original message:On Thu, May 7, 2015 at 12:12 AM, Matt Corallo <bitcoin-list at bluematt.me>
wrote:
> Recently there has been a flurry of posts by Gavin at
> http://gavinandresen.svbtle.com/ which advocate strongly for increasing
> the maximum block size. However, there hasnt been any discussion on this
> mailing list in several years as far as I can tell.
>
Thanks for bringing this up. I'll try to keep my arguments brief, to avoid
a long wall of text. I may be re-iterating some things that have been said
before, though.
I am - in general - in favor of increasing the size blocks: as technology
grows, there is no reason why the systems built on them can't scale
proportionally. I have so far not commented much about this, in a hope to
avoid getting into a public debate, but the way seems to be going now,
worries me greatly.
* Controversial hard forks. I hope the mailing list here today already
proves it is a controversial issue. Independent of personal opinions pro or
against, I don't think we can do a hard fork that is controversial in
nature. Either the result is effectively a fork, and pre-existing coins can
be spent once on both sides (effectively failing Bitcoin's primary
purpose), or the result is one side forced to upgrade to something they
dislike - effectively giving a power to developers they should never have.
Quoting someone: "I did not sign up to be part of a central banker's
committee".
* The reason for increasing is "need". If "we need more space in blocks" is
the reason to do an upgrade, it won't stop after 20 MB. There is nothing
fundamental possible with 20 MB blocks that isn't with 1 MB blocks.
Changetip does not put their microtransactions on the chain, not with 1 MB,
and I doubt they would with 20 MB blocks. The reason for increase should be
"because we choose to accept the trade-offs".
* Misrepresentation of the trade-offs. You can argue all you want that none
of the effects of larger blocks are particularly damaging, so everything is
fine. They will damage something (see below for details), and we should
analyze these effects, and be honest about them, and present them as a
trade-off made we choose to make to scale the system better. If you just
ask people if they want more transactions, of course you'll hear yes. If
you ask people if they want to pay less taxes, I'm sure the vast majority
will agree as well.
* Miner centralization. There is currently, as far as I know, no technology
that can relay and validate 20 MB blocks across the planet, in a manner
fast enough to avoid very significant costs to mining. There is work in
progress on this (including Gavin's IBLT-based relay, or Greg's block
network coding), but I don't think we should be basing the future of the
economics of the system on undemonstrated ideas. Without those (or even
with), the result may be that miners self-limit the size of their blocks to
propagate faster, but if this happens, larger, better-connected, and more
centrally-located groups of miners gain a competitive advantage by being
able to produce larger blocks. I would like to point out that there is
nothing evil about this - a simple feedback to determine an optimal block
size for an individual miner will result in larger blocks for better
connected hash power. If we do not want miners to have this ability, "we"
(as in: those using full nodes) should demand limitations that prevent it.
One such limitation is a block size limit (whatever it is).
* Ability to use a full node. I very much dislike the trend of people
saying "we need to encourage people to run full nodes, in order to make the
network more decentralized". Running 1000 nodes which are otherwise unused
only gives some better ability for full nodes to download the block chain,
or for SPV nodes to learn about transactions (or be Sybil-attacked...).
However, *using* a full node for validating your business (or personal!)
transactions empowers you to using a financial system that requires less
trust in *anyone* (not even in a decentralized group of peers) than
anything else. Moreover, using a full node is what given you power of the
systems' rules, as anyone who wants to change it now needs to convince you
to upgrade. And yes, 20 MB blocks will change people's ability to use full
nodes, even if the costs are small.
* Skewed incentives for improvements. I think I can personally say that I'm
responsible for most of the past years' performance improvements in Bitcoin
Core. And there is a lot of room for improvement left there - things like
silly waiting loops, single-threaded network processing, huge memory sinks,
lock contention, ... which in my opinion don't nearly get the attention
they deserve. This is in addition to more pervasive changes like optimizing
the block transfer protocol, support for orthogonal systems with a
different security/scalability trade-off like Lightning, making full
validation optional, ... Call me cynical, but without actual pressure to
work on these, I doubt much will change. Increasing the size of blocks now
will simply make it cheap enough to continue business as usual for a while
- while forcing a massive cost increase (and not just a monetary one) on
the entire ecosystem.
* Fees and long-term incentives. I put this last, not because I don't think
it is not serious, but because I don't understand nearly enough about it.
I'll let others comment.
I don't think 1 MB is optimal. Block size is a compromise between
scalability of transactions and verifiability of the system. A system with
10 transactions per day that is verifiable by a pocket calculator is not
useful, as it would only serve a few large bank's settlements. A system
which can deal with every coffee bought on the planet, but requires a
Google-scale data center to verify is also not useful, as it would be
trivially out-competed by a VISA-like design. The usefulness needs in a
balance, and there is no optimal choice for everyone. We can choose where
that balance lies, but we must accept that this is done as a trade-off, and
that that trade-off will have costs such as hardware costs, decreasing
anonymity, less independence, smaller target audience for people able to
fully validate, ...
Choose wisely.
Thanks for reading this,
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150507/29c77115/attachment.html>

2023-06-07 15:28:51

- reply

📅 Original date posted:2015-01-20
📝 Original message:Hello everyone,
We've been aware of the risk of depending on OpenSSL for consensus
rules for a while, and were trying to get rid of this as part of BIP
62 (malleability protection), which was however postponed due to
unforeseen complexities. The recent evens (see the thread titled
"OpenSSL 1.0.0p / 1.0.1k incompatible, causes blockchain rejection."
on this mailing list) have made it clear that the problem is very
real, however, and I would prefer to have a fundamental solution for
it sooner rather than later.
I therefore propose a softfork to make non-DER signatures illegal
(they've been non-standard since v0.8.0). A draft BIP text can be
found on:
https://gist.github.com/sipa/5d12c343746dad376c80
The document includes motivation and specification. In addition, an
implementation (including unit tests derived from the BIP text) can be
found on:
https://github.com/sipa/bitcoin/commit/bipstrictder
Comments/criticisms are very welcome, but I'd prefer keeping the
discussion here on the mailinglist (which is more accessible than on
the gist).
--
Pieter

2023-06-07 15:18:00

- reply

📅 Original date posted:2014-04-23
📝 Original message:On Wed, Apr 23, 2014 at 11:33 PM, Pavol Rusnak <stick at gk2.sk> wrote:
> On 04/23/2014 11:22 PM, Gregory Maxwell wrote:
>> Hopefully it would be clarified as only a MUST NOT do so silently...
>> "I have funds split across two wallets and it WONT LET ME SPEND THEM"
>> sounds like a terrible user experience. :)
>
> That is a subjective matter. To me it makes PERFECT SENSE that funds
> across accounts NEVER MIX. One can still send funds from one account to
> another and then perform another spend.
In that case, maybe it makes sense to define another purpose id
without accounts as well already.
I believe many simple wallets will find multiple subwallets too
burdening for the user experience, or not worth the technical
complexity.
--
Pieter

2023-06-07 15:17:59

- reply

📅 Original date posted:2014-04-23
📝 Original message:On Wed, Apr 23, 2014 at 10:43 PM, Pavol Rusnak <stick at gk2.sk> wrote:
> On 04/23/2014 10:41 PM, Luke-Jr wrote:
>> I don't see how. The user knows he has money in different subwallets. As long
>> as he has a way to specify which subwallet he is accessing in single-subwallet
>> clients, there shouldn't be a problem.
>
> Right. But these clients have no right to call themselves BIP64
> compatible then.
Would you consider software which scans all accounts as specified by
BIP64, but has no user interface option to distinguish them in any
way, view them independently, and has no ability to keep the coins
apart... compatible with BIP64?
According to the argument here mentioned earlier ("all or nothing"),
it is, as it will not break interoperability with other BIP64
software. Still, it doesn't support the accounts feature, and perhaps
that's fine?
--
Pieter

2023-06-07 15:16:19

- reply

📅 Original date posted:2014-03-27
📝 Original message:Just chiming in...
I'm not opposed to a more generic default key tree, but we need to
standardize this soon I believe. There are already existing code bases
that implement BIP32 wallets (and more are popping up...); just using
a separate one will result in lots of incompibilities.
That said, I'm not convinced about the extra layers. The "cointype" in
my opinion isn't necessary inside the derivation. There is already
support (4 bytes!) for magic bytes in the serialized form. Inside
applications/p2p it should always be known to which chain it applies,
and outside of that you shouldn't transfer raw keys. Maybe seeds need
some marker, but that's a separate case anyway. Mainnet and testnet
have specified magics here already - alts can define what they want
imho.
A 'reserved' field for future extensions may be useful, but as already
suggested by Mike, I don't believe we can encode how key chains are to
be used inside the derivation structure anyway. The most basic case
(not losing money in a wallet without special structure) can perhaps
be supported with just "the blockchain is your wallet", but I don't
believe this principle can scale to more advanced uses anyway, and you
need metadata in the wallet to deal with it.
In my view, your wallet just has a bunch of chains, and each chain
gets used for a particular purpose, fixing how the derivation beneath
it works. Either that is as a wallet, as part of a pair of multisig
keys, as a recurring payment receiver, ... or more complex things.
Some of these will require extra layers beneath, but that is
application specific. You would import a chain into your (advanced)
wallet with a particular extpub/extpriv code, and some metadata on how
to use it. Serialization formats for such designated extra uses sounds
better to me than trying to fit it into the derivation structure.
--
Pieter
On Thu, Mar 27, 2014 at 4:57 PM, Allen Piscitello
<allen.piscitello at gmail.com> wrote:
> Don't most of these coins have a magic number already assigned that is
> unique? (0xD9B4BEF9 for Bitcoin, 0x0709110B for Testnet, FBC0XB6DB for
> Litecoin, etc...). This seems like a good candidate for identifying coins,
> and also supports Testnet cases well. Maybe there are some alts without
> such a magic number that might prevent that?
>
> -Allen
>
>
> On Thu, Mar 27, 2014 at 10:43 AM, Jeff Garzik <jgarzik at bitpay.com> wrote:
>>
>> On Thu, Mar 27, 2014 at 3:09 AM, Tamas Blummer <tamas at bitsofproof.com>
>> wrote:
>> > A notable suggestion was to instead of building a directory of magic
>> > numbers
>> > (like 0 for Bitcoin, 1 for Litecoin etc) use a hash of the word
>> > "Bitcoin",
>> > "Litecoin", "Dogecoin", so collosion is unlikely and
>> > cetral directory is not needed.
>>
>> +1 good idea
>>
>> --
>> Jeff Garzik
>> Bitcoin core developer and open source evangelist
>> BitPay, Inc. https://bitpay.com/
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

2023-06-07 15:08:01

- reply

📅 Original date posted:2013-10-26
📝 Original message:Let's first try to agree on what we are solving.
It seems that Thomas wants - in addition to the cryptographic data -
to encode the tree structure, or at least version information about
what features are used in it, inside the seed.
I'm not sure whether we're ready to standardize on something like that
yet, not having established best practices regarding different wallet
structures. I do think we first need to see what possibilities and
developments are possible related to it.
In addition, information about the wallet structure is strictly less
secret than the key data - it is even less confidential than address
book data, transaction annotations, labels and comments and
bookkeeping information. It could be backed up anywhere and everywhere
without any repercussions, as far as I can see. I understand that in
the case of Electrum, there is a strong reason to want this
encapsulated together with the seed, but it's not really a requirement
for most wallets.
(if really needed, any key derivation scheme that starts from random
strings can be augmented with metadata by enforcing property-bits on a
hash of the string (so not of the output), as this data doesn't need
protection from brute-forcing).
Regarding other requirements, I wonder why we want the transformation
to be bidirectional? If it is just about generating master seeds, one
direction should be enough, and allows far nicer properties w.r.t.
security. If we do settle on a standard for 'brainwallets', I would
strongly prefer if it had at least some strengthening built-in, to
decrease the impact of worst-case situations.
If the reason is backward-compatibility, I think that any client that
supports seeds already can just keep supporting whatever they
supported before. Only if it matches both encoding schemes (as
mentioned before) there is a potential collision (and in that case,
the user could just be asked).
--
Pieter
On Sat, Oct 26, 2013 at 10:47 PM, slush <slush at centrum.cz> wrote:
> Hi Thomas,
>
> can you more elaborate on that "version" bits? What is exact meaning of it?
> I still think this is more an implementation problem. What stops Electrum to
> do the same algorithm for searching branches as it is now for used
> addresses?
>
> These "version bits" need to be covered by the specification as well,
> because if any client will use them differently (or won't use them at all),
> it will break cross-compatibility between clients, which was another goal of
> bip39.
>
> Marek
>
>
>
>
> On Sat, Oct 26, 2013 at 5:24 PM, Thomas Voegtlin <thomasv1 at gmx.de> wrote:
>>
>> here is a simple implementation, with some ideas on how to format the
>> metadata:
>> https://en.bitcoin.it/wiki/Talk:BIP_0039
>>
>>
>>
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>> from
>> the latest Intel processors and coprocessors. See abstracts and register >
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

2023-06-07 15:01:34

- reply

📅 Original date posted:2013-05-09
📝 Original message:On Wed, May 08, 2013 at 10:42:44PM -0400, Peter Todd wrote:
> Ah, shoot, I just realized we both got missed Pieter's point entirely:
> he means to change the meaning of the header timestamp to be relative
> time passed since the last block...
No, though that's also a possibility, but a backward-incompatible one.
What I mean is have a well-defined 64-bit timestamp for each block, but
only put the lowest 32 bit in the header. Under the condition:
* There is never a gap of more than 136 years between two blocks.
The actual 64-bit timestamp can be deterministically derived from the
header, by prefixing it with the lowest 32-bit value that does not
cause the result to violate the
at-least-above-the-median-of-the-previous-11-blocks rule.
--
Pieter

2023-06-07 15:01:32

- reply

📅 Original date posted:2013-05-08
📝 Original message:On Wed, May 08, 2013 at 09:08:34PM -0400, Jeff Garzik wrote:
> On Wed, May 8, 2013 at 9:00 PM, John Dillon
> <john.dillon892 at googlemail.com> wrote:
> > Perhaps Satoshi did this delibrately, knowing that at some point a hard-fork
> > would be a good idea, so that we all would have a good excuse to do one?
>
> Guffaw :) The year 2038 is so far in the future that it is not really
> relevant, from that angle.
"Meh". I think it's highly unlikely we'll break the block header format, as it
pretty much means invalidating all mining hardware.
There's also no need: 32 bits is plenty of precision. Hell, even 16 bits would
do (assuming there's never more than a 65535s (about 18 hours) gap between two
blocks). Just assume the "full" 64-bit time is the smallest one that makes
sense, given its lower 32 bits.
--
Pieter

2023-06-07 15:01:29

- reply

📅 Original date posted:2013-05-07
📝 Original message:On Tue, May 07, 2013 at 02:16:41PM +0200, Adam Back wrote:
> Hi
>
> Three minor security/other issues:
>
> 1. please a way to unlock the wallet without displaying wallet password in
> console screen (console unlock wallet, to import priv key); or
I think the general solution here is providing a feature-reach Python RPC client,
which can do things like remember passwords, command history/tab completion,
perhaps even batch lookups of compound commands (getblock $(getblockhash X, for
example, ...). The naive RPC client built into bitcoind is not a good fit for
many features, as they can much more efficiently be developed outside of the
core binary,
> 2. a button to import a private key (and option to transfer it to another
> key - if you are not the sole controller the private key)
I'm quite opposed to any per-key fiddling in the GUI. This will inevitably lead
to (even more) people misunderstanding how wallets work and shooting themself in
the foot. I don't mind an expert mode ("coin control") that enables such features,
but in general, we should for entire-wallet export and import rather than
individual keys.
Import & sweep an address is something else, that sounds safe to.
> 3. a UX way to transfer BTC off a specific adress (eg choose from
> address), rather than having to spend the entire wallet onto a new
> address, just to get BTC off a specific address. Doing it that way has
> problems: creates more network traffic/bigger packets, higher fees (if
> any transactions are young/low confirmation), and generally damages
> privacy as all your funds end up linked.
This belongs in coin control, IMHO.
--
Pieter

2023-06-07 15:01:11

- reply

📅 Original date posted:2013-05-06
📝 Original message:On Mon, May 06, 2013 at 10:19:35AM +0200, Mike Hearn wrote:
> You are welcome to optimise P2P addr broadcasts or develop better bootstrap
> mechanisms.
I think John's actually has a point here. If we're judging the quality of a
protocol change by how compatible it is with DNS seeding, then we're clearly not
using DNS seeding as seeding anymore (=getting an entry point into the P2P
network), but as a mechanism for choosing (all) peers.
Eventually, I think it makes sense to move to a system where you get seeds from
a DNS (or other mechanism), connect to one or a few of the results, do a getaddr,
fill your peer IP database with it, and disconnect from the DNS seeded peer.
This probably means we need to look at ways to optimize current peer exchange,
but that's certainly welcome in any case.
--
Pieter

2023-06-07 14:56:18

- reply

📅 Original date posted:2013-05-03
📝 Original message:(generic comment on the discussion that spawned off: ideas about how to
allow additional protocols for block exchange are certainly interesting,
and in the long term we should certainly consider that. For now I'd like to
keep this about the more immediate way forward with making the P2P protocol
not break in the presence of pruning nodes)
On Sun, Apr 28, 2013 at 6:57 PM, Mike Hearn <mike at plan99.net> wrote:
> That's true. It can be perhaps be represented as "I keep the last N
> blocks" and then most likely for any given node the policy doesn't change
> all that fast, so if you know the best chain height you can calculate which
> nodes have what.
>
Yes, I like that better than broadcasting the exact height starting at
which you serve (though I would put that information immediately in the
version announcement). I don't think we can rely on the addr broadcasting
mechanism for fast information exchange anyway. One more problem with this:
DNS seeds cannot convey this information (neither do they currently convey
service bits, but at least those can be indexed separately, and served
explicitly through asking for a specific subdomain or so).
So to summarize:
* Add a field to addr messages (after protocol number increase) that
maintains number of top blocks served)?
* Add a field to version message to announce the actual first block served?
* Add service bits to separately enable "relaying/verifying node" and
"serves (part of) the historic chain"? My original reason for suggesting
this was different, I think better compatibility with DNS seeds may be a
good reason for this. You could ask the seed first for a subset that at
least serves some part of the historic chain, until you hit a node that has
enough, and once caught up, ask for nodes that relay.
Disconnecting in case something is requested that isn't served seems like
>> an acceptable behaviour, yes. A specific message indicating data is pruned
>> may be more flexible, but more complex to handle too.
>>
>
> Well, old nodes would ignore it and new nodes wouldn't need it?
>
I'm sure there will be cases where a new node connects based on outdated
information. I'm just stating that I agree with the generic policy of "if a
node requests something it should have known the peer doesn't serve, it is
fair to be disconnected."
> The reason for splitting them is that I think over time these may be
>> handled by different implementations. You could have stupid
>> storage/bandwidth nodes that just keep the blockchain around, and others
>> that validate it. Even if that doesn't happen implementation-wise, I think
>> these are sufficiently independent functions to start thinking about them
>> as such.
>>
>
> Maybe so, with a "last N blocks" in addr messages though such nodes could
> just set their advertised history to zero and not have to deal with serving
> blocks to nodes.
>
> If you have a node that serves the chain but doesn't validate it, how does
> it know what the best chain is? Just whatever the hardest is?
>
Maybe it validates, maybe it doesn't. What matters is that it doesn't
guarantee relaying fresh blocks and transactions. Maybe it does validate,
maybe it just stores any blocks, and uses a validating node to know what to
announce as best chain, or it uses an SPV mechanism to determine that. Or
it only validates and relays blocks, but not transactions. My point is that
"serving historic data" and "relaying fresh data" are separate
responsibilities, and there's no need to require them to be combined.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20130503/9a6e4327/attachment.html>

2023-06-07 11:31:51

- reply

📅 Original date posted:2013-02-09
📝 Original message:On 9 Feb 2013 00:50, "Gavin Andresen" <gavinandresen at gmail.com> wrote:
> Then before final release (or rc2, if that is needed) pull #2286 and
> create and pull a patch to fix the windows non-determinism problem.
#2243 should fix that, I think.
--
Pieter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20130209/8416e7ec/attachment.html>

2023-06-07 11:30:37

- reply

📅 Original date posted:2013-01-27
📝 Original message:On Sun, Jan 27, 2013 at 05:27:52AM -0500, Peter Todd wrote:
> seed.txt? You mean the dumpfile produced by bitcoin-seeder? That has
> uptime info, although only a months worth if I understand it correctly.
It has information about all non-banned IPs that were ever connected to
succesfully. The connectability% is a decaying average, with a time
constant of up to a month, though.
I have a script ready which combines one or more seeds.txt's data into a
pnSeed[] array C source code, by the way. I'll add it to the repository.
--
Pieter

2023-06-07 10:04:23

- reply

📅 Original date posted:2012-04-14
📝 Original message:On Sat, Apr 14, 2012 at 04:20:50PM -0400, Jeff Garzik wrote:
> >> Furthermore, many of these ideas -- like sending TX's directly to the
> >> merchant -- involve far more direct payee<->payer communication on the
> >> part of the wallet client than is currently envisioned
> >
> > Yes, though it's worth remembering that the original Bitcoin design
> > did have participants communicate directly. When I talked with Satoshi
> > in 2009 he saw the pay-to-IP-address mode imagined as the normal way
> > to make payments, with pay-to-address being used as a kind of backup
> > for when the recipient was offline.
> >
> > In the end that's not how things evolved, but it the pendulum could
> > easily swing back the other way.
>
> But I also have a "gut feeling" that these sorts of payments and
> direct communication should be done via a wholly separate protocol
> than the bitcoin P2P protocol. Doing p2ip as it was done originally,
> inside the bitcoin P2P protocol, was a mistake. Extensible as it is,
> I think a better job -- and faster evolution -- can be done with a
> separate protocol on a separate port.
>
> Some HTTP derivative would probably make life easier for mobile
> payments and firewalled scenarios, and for client->merchant
> communications, for instance.
Have you ever read https://gist.github.com/1237788 ?
--
Pieter

2023-06-07 10:03:29

- reply

📅 Original date posted:2012-04-12
📝 Original message:On Thu, Apr 12, 2012 at 11:41:05AM -0400, Gavin Andresen wrote:
> On Wed, Apr 11, 2012 at 2:39 PM, Christian Bodt <sirk390 at gmail.com> wrote:
> > I would like to discuss the following bitcoin protocol improvement proposal:
> >
> > Adding request/reply id in all messages (in the message header,
> > based on what was done for the "checksum" field)
>
> That seems like a perfectly reasonable protocol improvement to me.
> Anybody else have an opinion?
If there is a reasonable use for it, I have no objections.
However: the bitcoin P2P protocol is not fully request-reply based, and trying to use
it that may be be less intuitive than how it looks. For example, doing a second
identical "getblocks" request will not result in more "inv" replies, as the client
prevents retransmits. This is not a large problem, but maybe such an extension
should also include an extra "denied" message, which is sent if the client is
unwilling to answer (and may also be used to report transactions that are not
accepted into the memory pool, for example).
--
Pieter

2023-06-07 10:02:12

- reply

📅 Original date posted:2012-04-02
📝 Original message:Hello all,
Mike Hearn has submitted a pull request to add a pong message in reply to a ping.
This warrants an upgrade of the network protocol version number, which is since BIP14
independent from the version numbers of the reference client.
Any opinions about a numbering scheme? Currently the value 60000 is used. We could
simply start increasing the number one by one now for every change, or we could
do a "cleanup" to 100000 first, and start incrementing from there.
--
Pieter

2023-06-07 02:47:35

- reply

📅 Original date posted:2011-12-16
🗒️ Summary of this message: Temporary addresses for anonymity require interaction with the wallet-hosting entity. EC crypto tricks can generate fresh private keys for each transaction.
📝 Original message:On Thu, Dec 15, 2011 at 04:26:38PM +0800, Walter Stanish wrote:
> Interaction is a requirement, since there seems to be a widely felt
> need to preserve anonymity through the use of temporary addresses.
> Generating a temporary address requires some actual processing to
> achieve, since the issuing of the new address cannot be done without
> interacting with the entity hosting the wallet (unless I'm missing
> something?).
Just replying to this one comment: yes, some interaction is always
necessary, but not necessarily directly with the entity hosting the wallet.
There are some EC crypto tricks to do this (often mentioned under
"deterministic wallets" before):
The wallet-hosting entity has a private key x, with public key X.
The address-generating entity knows X, and generates a fresh private
key y for each transaction. For each, it calculates Z=y*X, and asks
the client to pay to hash160(Z). Afterwards, it can send a bunch of
y's to the wallet hosting service, which can reconstruct z=y*x for
each. Alternatively, the y's can be generated according to a predefined
scheme instead.
--
Pieter

2023-06-07 02:44:07

- reply

📅 Original date posted:2011-12-18
🗒️ Summary of this message: Using DNS-based alias systems for Bitcoin payments is vulnerable to spoofing. One solution is to embed a Bitcoin address in the identification string itself.
📝 Original message:On Mon, Dec 19, 2011 at 12:58:37AM +0100, slush wrote:
> Maybe I'm retarded, but where's the point in providing alliases containing
> yet another hash in URL?
Any DNS-based alias system is vulnerable to spoofing. If I can make people's
DNS server believe that mining.cz points to my IP, I'll receive payments to
you...
If no trusted CA is used to authenticate the communication, there is no way
to be sure the one you are asking how to pay, is the person you want to pay.
Therefore, one solution is to put a bitcoin address in the identification
string itself, and requiring SSL communication authenticated using the
respective key.
This makes the identification strings obviously less useful as aliases,
but pure aliases in the sense of human-typable strings have imho
limited usefulness anyway - in most cases these identification strings
will be communicated through other electronic means anyway.
Furthermore, the embedded bitcoin address could be hidden from the user:
retrieved when first connecting, and stored together with the URI in
an address book. Like ssh, it could warn the user if the key changes
(which wil be ignored by most users anyway, but what do you do about
that?)
--
Pieter

2023-06-07 01:22:02

- reply

📅 Original date posted:2011-06-22
🗒️ Summary of this message: Developers are discussing merging autotools and qtgui, but autotools needs more testing on native Windows and OSX platforms. Testing requested.
📝 Original message:On Sun, Jun 19, 2011 at 02:30:49AM +0200, Pieter Wuille wrote:
> The consensus seems to be to merge autotools and qtgui. However, autotools is
> apparently not tested (enough) on native windows and osx platforms.
>
> Anyone on this list with either development environnement, and willingness
> to test?
Jaromil has his autotools branch rebased against git master, and included Mark's
NOSIGPIPE/NOSIGNAL patch (needs a commit summary, though).
Can people test this compiles+works on several systems (ubuntu, fedora(?), mingw
crosscompile, osx, ...)?
See https://github.com/jaromil/bitcoin/commits/autotools3
For me, it works on Ubuntu 10.10 amd64, including wx gui.
--
Pieter

2023-06-07 01:21:58

- reply

📅 Original date posted:2011-06-18
🗒️ Summary of this message: John Smith requests information on the roadmap for including the autotools build system to support multiple GUIs in Qt. Pieter suggests testing on native Windows and OSX platforms.
📝 Original message:On Fri, Jun 17, 2011 at 06:42:30AM +0000, John Smith wrote:
> Hello,
>
> I think my Qt GUI is pretty much ready for merging. I've been using it for a
> while and all functionality works.
>
> What is the current roadmap for including the autotools build system, so
> that it will be possible to support multiple GUIs? 0.4.0?
The consensus seems to be to merge autotools and qtgui. However, autotools is
apparently not tested (enough) on native windows and osx platforms.
Anyone on this list with either development environnement, and willingness
to test?
--
Pieter

2023-06-07 01:21:39

- reply

📅 Original date posted:2011-06-17
🗒️ Summary of this message: Pieter suggests merging the autotools build system and the Qt GUI, as long as they are stable and marked as experimental.
📝 Original message:On Fri, Jun 17, 2011 at 06:42:30AM +0000, John Smith wrote:
> Hello,
>
> I think my Qt GUI is pretty much ready for merging. I've been using it for a
> while and all functionality works.
>
> What is the current roadmap for including the autotools build system, so
> that it will be possible to support multiple GUIs? 0.4.0?
I'm in favor of merging autotools now, unless there are still known issues with
it, such as mingw32 compatibility?
If the Qt port is functionally complete and stable, I'd vote to have it merged as
well under these conditions:
* the changes to the 'core' are minimal/trivial
* marked as 'experimental', and not enabled by default
The only disadvantage of another GUI is that the (necessary) cleanup of RPC/GUI
code will now need makes changes in 3 places instead of 2, but as I understand
it, there are much more people willing to work on Qt code than on wx code?
--
Pieter