Karl Johan Alm [ARCHIVE]

⚠️ This account is not associated with a real person. It's an archival account for mailing list messages.
🗓️ Active from 2017-05-08 to 2018-04-04
💬 Posted a total of 19 messages
📚 Participated in the following categories: bitcoin-dev
🔇 Please do not DM this user as it's used for archive purposes only.

Public Key

npub1e7vdq905zr4xjr5nxuz58l9jcvffxq728ysl6m2xxgr024mjy5vq4fg307

Profile Code

nprofile1qqsvlxxszh6pp6nfp6fnwp2rljevxy5nq09rjg0ad4rryph42aez2xqprpmhxue69uhhyetvv9ujumn0wdmksetjv5hxxmmdqyt8wumn8ghj7un9d3shjtnwdaehgu3wvfskueqhfuc3q

Publishing to

relay.noswhere.com

relay.nostr.band

Author Public Key

npub1e7vdq905zr4xjr5nxuz58l9jcvffxq728ysl6m2xxgr024mjy5vq4fg307

Show more details

Last Notes

2023-06-07 18:11:31

- reply

📅 Original date posted:2018-04-04
📝 Original message:I took the liberty of turning this into a BIP proposal -- the
formatted version can be seen here:
https://github.com/kallewoof/bips/blob/bip-typed-wif/bip-extended-privkey.mediawiki
<pre>
BIP: XXX
Layer: Applications
Title: Typed Private Keys
Author: Karl-Johan Alm <karljohan-alm at garage.co.jp>
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-XXX
Status: Draft
Type: Standards Track
Created: 2018-04-04
License: CC0-1.0
</pre>
== Abstract ==
An extension to the private key (WIF) format to specify what kind of
public key the private key corresponds to.
== Motivation ==
There are several types of public keys which can all be associated
with a given private key: P2PKH (legacy <code>1...</code> format),
P2SH-P2WPKH (SegWit public key inside P2SH), P2WPKH (bech32), etc.
While private keys have a 1-byte suffix indicating whether the
corresponding public key is compressed (<code>0x01</code>) or not
(<code>0x00</code>), there is no way of knowing what kind of public
keys were associated with the private key. As a result, when importing
a private key, the wallet has to assume all kinds, and keep track of
each possible alternative.
By extending the suffix, we can specify what kind of public key was
associated with the given private key.
== Specification ==
Currently, private keys are stored as a uint256 (private key data)
followed by a uint8 (compressed flag). The latter is extended to
specify the public key types:
{|class="wikitable" style="text-align: center;"
|-
!Value
!Type
!Compr
!Clarification
|-
|<code>0x00</code>||P2PKH_UNCOMPRESSED||No||Uncompressed legacy public
key. Unknown public key format
|-
|<code>0x01</code>||P2PKH_COMPRESSED||Yes||Compressed legacy public
key. Unknown public key format
|-
|<code>0x80</code>||P2PKH||Yes||Compressed legacy public key. Legacy
public key format (<code>1...</code>)
|-
|<code>0x81</code>||P2WPKH||Yes||Bech32 format (native Segwit)
|-
|<code>0x82</code>||P2WPKH_P2SH||Yes||Segwit nested in BIP16 P2SH
(<code>3...</code>)
|-
|<code>0x85</code>||P2SH|| — ||Non-Segwit BIP16 P2SH (<code>3...</code>)
|-
|<code>0x86</code>||P2WSH|| — ||Native Segwit P2SH
|-
|<code>0x87</code>||P2WSH_P2SH|| — ||Native Segwit P2SH nested
in BIP16 P2SH
|}
When a wallet imports a private key, it will have two outcomes:
* the key is using one of the legacy types, in which case all types
must be accounted for
* the key is using one of the extended types, in which case the wallet
need only track the specific corresponding public key
== Rationale ==
TODO
== Compatibility ==
This proposal is not backwards compatible, in that software that does
not recognize the new types will not understand the compressed flag.
It would be trivial to change this, by keeping the 'uncompressed'
state as it is (0) and changing 'compressed' to be 'anything not 0',
as opposed to 'the value 1'.
The proposal *is* backwards compatible in that new wallet software
will always understand the old WIF format, however. It will, as it
does today, assume that any kind of public key is possible, and will
have to track all of them, as it has to today.
== Acknowledgements ==
This BIP is based on the initial proposal by Thomas Voegtlin
<thomasv at electrum.org> on the Bitcoin Dev mailing
list<ref>https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015007.html</ref>
and the Electrum 3.0
implementation<ref>https://github.com/spesmilo/electrum/blob/82e88cb89df35288b80dfdbe071da74247351251/RELEASE-NOTES#L95-L108</ref>
== Reference implementation ==
There is a partial implementation which adds, but does not use, the
types described in this BIP here:
https://github.com/bitcoin/bitcoin/pull/12869
== References ==
<references/>
== Copyright ==
This document is licensed under the Creative Commons CC0 1.0 Universal license.
On Mon, Sep 18, 2017 at 12:36 AM, Mark Friedenbach via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> Bech32 and WIF payload format are mostly orthogonal issues. You can design a
> new wallet import format now and later switch it to Bech32.
>
> On Sep 17, 2017, at 7:42 AM, AJ West via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Hi I have a small interjection about the point on error correction (excuse
> me if it seems elementary). Isn't there an argument to be made where a
> wallet software should never attempt to figure out the 'correct' address, or
> in this case private key? I don't think it's crazy to suggest somebody could
> import a slightly erroneous WIF, the software gracefully error-corrects any
> problem, but then the user copies that error onward such as in their backup
> processes like a paper wallet. I always hate to advocate against a feature,
> I'm just worried too much error correcting removes the burden of exactitude
> and attention of the user (eg. "I know I can have up to 4 errors").
>
> I'm pretty sure I read those arguments somewhere in a documentation or issue
> tracker/forum post. Maybe I'm misunderstanding the bigger picture in this
> particular case, but I was just reminded of that concept (even if it only
> applies generally).
>
> Thanks,
> AJ West
>
> On Sun, Sep 17, 2017 at 4:10 AM, Thomas Voegtlin via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>> On 17.09.2017 04:29, Pieter Wuille wrote:
>> >
>> > This has been a low-priority thing for me, though, and the computation
>> > work
>> > to find a good checksum is significant.
>> >
>>
>> Thanks for the info. I guess this means that a bech32 format for private
>> keys is not going to happen soon. Even if such a format was available,
>> the issue would remain for segwit-in-p2sh addresses, which use base58.
>>
>> The ambiguity of the WIF format is currently holding me from releasing a
>> segwit-capable version of Electrum. I believe it is not acceptable to
>> use the current WIF format with segwit scripts; that would just create
>> technological debt, forcing wallets to try all possible scripts. There
>> is a good reason why WIF adds a 0x01 byte for compressed pubkeys; it
>> makes it unambiguous.
>>
>> I see only two options:
>> 1. Disable private keys export in Electrum Segwit wallets, until a
>> common WIF extension has been agreed on.
>> 2. Define my own WIF extension for Electrum, and go ahead with it.
>>
>> Defining my own format does make sense for the xpub/xprv format, because
>> Electrum users need to share master public keys across Electrum wallets.
>> It makes much less sense for WIF, though, because WIF is mostly used to
>> import/sweep keys from other wallets.
>>
>> I would love to know what other wallet developers are going to do,
>> especially Core. Are you going to export private keys used in segwit
>> scripts in the current WIF format?
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

2023-06-07 18:11:16

- reply

📅 Original date posted:2018-03-15
📝 Original message:On Thu, Mar 15, 2018 at 6:43 AM, Jim Posen <jim.posen at gmail.com> wrote:
> How are scripts with OP_CLTV and OP_CSV handled by verifiers? Do they always
> succeed? Or should an nLockTime and nSequence also be included in the proof
> in a way that can be parsed out and displayed to verifiers?
Good question.. Since you don't really have the input(s), I think it's
fine to always assume sufficient time/height on CLTV/CSV checks.
> I assume any signatures in the scriptSig/witness data would have no sighash
> type?
I think it would just use the default (SIGHASH_ALL?) for simplicity.
Is there a good reason to tweak it?
-Kalle.

2023-06-07 18:11:15

- reply

📅 Original date posted:2018-03-16
📝 Original message:On Fri, Mar 16, 2018 at 1:59 AM, Greg Sanders <gsanders87 at gmail.com> wrote:
> Sorry if I missed the rationale earlier, but why not just do a transaction,
> with a FORKID specifically for this? Then a node can have a mempool
> acceptance test that returns true even if the signature is not valid as per
> Bitcoin consensus, but only due to the FORKID?
>
> This way basically any wallet can support this provided generic FORKID
> support.
You'd basically have to provide an entire transaction rather than just
the signature, so there's some overhead. (Copy-pasting may become
unwieldy quicker.)

2023-06-07 18:11:15

- reply

📅 Original date posted:2018-03-27
📝 Original message:Pieter,
Thanks for the feedback. Comments below:
On Mon, Mar 26, 2018 at 5:53 PM, Pieter Wuille <pieter.wuille at gmail.com> wrote:
> One solution is to include a version number in the signature, which
> explicitly corresponds to a set of validation flags. When the version number
> is something a verifier doesn't know, it can be reported as inconclusive
> (it's relying on features you don't know about).
>
> An solution is to verify twice; once with all consensus rules you know
> about, and once with standardness rules. If they're both valid, the
> signature is valid. If they're both invalid, the signature is invalid. If
> they're different (consensus valid but standardness invalid), you report the
> signature validation as inconclusive (it's relying on features you don't
> know about). This approach works as long as new features only use previous
> standardness-invalid scripts, but perhaps a version number is still needed
> to indicate the standardness flags.
I think the double verify approach seems promising. I assume old nodes
consider new consensus rule enforcing transactions as non-standard but
valid. If this is always the case, it may be an idea to simply fail
verification with a message indicating the node is unable to verify
due to unknown consensus rules.
>> RPC commands:
>>
>> sign <address> <message> [<prehashed>=false]
>
> Why not extend the existing signmessage/verifymessage RPC? For legacy
> addresses it can fall back to the existing signature algorithm, while using
> the script-based approach for all others.
Yes, I initially thought it would be better to not use it as the
legacy behavior could be depended on god knows where, but I think
adding a legacy mode or simply doing the old way for 1xx is
sufficient.
>> (**) If <prehashed> is true, <message> is the sighash, otherwise
>> sighash=sha256d(message).
>
>
> That's very dangerous I'm afraid. It could be used to trick someone into
> signing off on an actual transaction, if you get them to sign a "random
> looking" prehashed message. Even if you have a prehashed message, there is
> no problem with treating it as hex input to a second hashing step, so I
> think the prehashed option isn't needed. It's why the existing message
> signing functionality always forcibly prefixes "Bitcoin signed message:", to
> avoid signing something that unintentionally corresponds to a message
> intended for another goal.
Eek.. good point...

2023-06-07 18:11:14

- reply

📅 Original date posted:2018-03-15
📝 Original message:On Thu, Mar 15, 2018 at 2:14 PM, Luke Dashjr <luke at dashjr.org> wrote:
> Not necessarily specific UTXOs (that would contradict fungibility, as well as
> be impossible for hot/cold wallet separation), but just to prove funds are
> available. The current sign message cannot be used to prove present possession
> of funds, only that you receive funds.
By saying "not necessarily specific UTXOs", are you saying it may be
spent outputs? I'm a little confused I think.
On Thu, Mar 15, 2018 at 8:53 PM, Jim Posen <jim.posen at gmail.com> wrote:
> In this general signing-a-script context, I think a verifier might want to
> see the time conditions under which it may be spent. The proof container
> could include an optional nLockTime which defaults to 0 and nSequence which
> defaults to 0xFFFF...
Good point!
>> I think it would just use the default (SIGHASH_ALL?) for simplicity.
>> Is there a good reason to tweak it?
>
> I took another look and there should definitely be a byte appended to the
> end of the sig so that the encoding checks pass, but I think it might as
> well be a 0x00 byte since it's not actually a sighash flag.
I think the sighash flag affects the outcome of the actual
verification, but I could be mistaken.
-Kalle.

2023-06-07 18:11:14

- reply

📅 Original date posted:2018-03-15
📝 Original message:On Wed, Mar 14, 2018 at 12:36 PM, Luke Dashjr <luke at dashjr.org> wrote:
> Ideally, it should support not only just "proof I receive at this address",
> but also "proof of funds" (as a separate feature) since this is a popular
> misuse of the current message signing (which doesn't actually prove funds at
> all). To do this, it needs to be capable of signing for multiple inputs.
Re-reading this, I think what you mean is it should be possible to
create a proof for (a) specific UTXO(s), hence "inputs". That sounds
pretty useful, yeah!
So you could provide a mix of addresses and inputs (as txid:vout) and
it would generate a proof that signs the message for each input
(taking scriptPubKey from address or from the UTXO data directly on
the blockchain).
-Kalle.

2023-06-07 18:11:13

- reply

📅 Original date posted:2018-03-15
📝 Original message:On Wed, Mar 14, 2018 at 5:46 AM, Kalle Rosenbaum <kalle at rosenbaum.se> wrote:
> I can't really see from your proposal if you had thought of this: A soft
> fork can make old nodes accept invalid message signatures as valid. For
> example, a "signer" can use a witness version unknown to the verifier to
> fool the verifier. Witness version is detectable (just reject unknown
> witness versions) but there may be more subtle changes. Segwit was not
> "detectable" in that way, for example.
>
> This is the reason why I withdrew BIP120. If you have thought about the
> above, I'd be very interested.
I'm not sure I see the problem. The scriptPubKey is derived directly
from the address in all cases, which means the unknown witness version
would have to be committed to in the address itself.
So yeah, I can make a P2SH address with a witness version > 0 and a to
me unknown pubkey and then fool you into thinking I own it, but I
don't really see why you'd ultimately care. In other words, if I can
SPEND funds sent to that address today, I can prove that I can spend
today, which is the purpose of the tool, I think.
For the case where the witness version HAS been upgraded, the above
still applies, but I'm not sure it's a big issue. And it doesn't
really require an old node. I just need to set witness version >
current witness version and the problem applies to all nodes.
On Wed, Mar 14, 2018 at 8:36 AM, Luke Dashjr <luke at dashjr.org> wrote:
> I don't see a need for a new RPC interface, just a new signature format.
All right.
> Ideally, it should support not only just "proof I receive at this address",
> but also "proof of funds" (as a separate feature) since this is a popular
> misuse of the current message signing (which doesn't actually prove funds at
> all). To do this, it needs to be capable of signing for multiple inputs.
I assume by inputs you mean addresses/keys. The address field could
optionally be an array. That'd be enough?
> Preferably, it should also avoid disclosing the public key for existing or
> future UTXOs. But I don't think it's possible to avoid this without something
> MAST-like first. Perhaps it can be a MAST upgrade later on, but the new
> signature scheme should probably be designed with it in mind.
I'd love to not have to reveal the public key, but I'm not sure how it
would be done, even with MAST.
On Wed, Mar 14, 2018 at 12:12 PM, Anthony Towns <aj at erisian.com.au> wrote:
> Wouldn't it be sufficient for old nodes to check for standardness of the spending script and report non-standard scripts as either invalid outright, or at least highly questionable? That should prevent confusion as long as soft forks are only making nonstandard behaviours invalid.
That seems sensible to me. A warning would probably be useful, in case
the verifier is running old software.
-Kalle.

2023-06-07 18:11:12

- reply

📅 Original date posted:2018-03-14
📝 Original message:Hello,
I am considering writing a replacement for the message signing tools
that are currently broken for all but the legacy 1xx addresses. The
approach (suggested by Pieter Wuille) is to do a script based
approach. This does not seem to require a lot of effort for
implementing in Bitcoin Core*. Below is my proposal for this system:
A new structure SignatureProof is added, which is a simple scriptSig &
witnessProgram container that can be serialized. This is passed out
from/into the signer/verifier.
RPC commands:
sign <address> <message> [<prehashed>=false]
Generates a signature proof for <message> using the same method that
would be used to spend coins sent to <address>.**
verify <address> <message> <proof> [<prehashed>=false]
Deserializes and executes the proof using a custom signature checker
whose sighash is derived from <message>. Returns true if the check
succeeds, and false otherwise. The scriptPubKey is derived directly
from <address>.**
Feedback welcome.
-Kalle.
(*) Looks like you can simply use VerifyScript with a new signature
checker class. (h/t Nicolas Dorier)
(**) If <prehashed> is true, <message> is the sighash, otherwise
sighash=sha256d(message).

2023-06-07 18:07:22

- reply

📅 Original date posted:2017-11-01
📝 Original message:This is the paper detailing the research behind my talk "Optimizing
fee estimation via the mempool state" (the presentation only covers
part of the paper) at Scaling Stanford (this coming Sunday). Feedback
welcome.
https://bc-2.jp/mempool.pdf

2023-06-07 18:05:36

- reply

📅 Original date posted:2017-09-12
📝 Original message:On Wed, Sep 13, 2017 at 4:57 AM, Mark Friedenbach via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
>> Without the limit I think we would be DoS-ed to dead
>
> 4MB of secp256k1 signatures takes 10s to validate on my 5 year old
> laptop (125,000 signatures, ignoring public keys and other things that
> would consume space). That's much less than bad blocks that can be
> constructed using other vulnerabilities.
Sidenote-ish, but I also believe it would be fairly trivial to keep a
per UTXO tally and demand additional fees when trying to respend a
UTXO which was previously "spent" with an invalid op count. I.e. if
you sign off on an input for a tx that you know is bad, the UTXO in
question will be penalized proportionately to the wasted ops when
included in another transaction later. That would probably kill that
DoS attack as the attacker would effectively lose bitcoin every time,
even if it was postponed until they spent the UTXO. The only thing
clients would need to do is to add a fee rate penalty ivar and a
mapping of outpoint to penalty value, probably stored as a separate
.dat file. I think.

2023-06-07 18:05:36

- reply

2023-06-07 18:04:22

- reply

📅 Original date posted:2017-07-11
📝 Original message:On Wed, Jul 12, 2017 at 6:11 AM, Gregory Maxwell via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> IMO the way to do "roadmaps" in Bitcoin is to roadmap the finalization
> and release process once the basic technology is done; because it's
> only past that point that guarantees can really start being made.
Bitcoin development differs from Linux kernel development in a number
of obvious ways, such as the fact Bitcoin is being "patched in
flight". The current political situation over Bitcoin development is
also quite different, with scalability being a major concern for a lot
of users, and conflicting views leading to risky technical gambles.
Having *something* like a roadmap that gives the average user some
insights into what exactly is being planned for Bitcoin is very
desirable, arguably even necessary, in particular for the scaling
solutions. Putting deadlines and dates in would of course be highly
irresponsible, as no one can predict how much of their free time
volunteer developers will put into the project in advance (or whether
they will stick around for the next X months or stop being
contributors).
I think there is necessity for a document that describes the project
intentions for scaling solutions, but I don't think adding dates and
deadlines is appropriate. That may or may not be a roadmap. I imagine
such a document would be updated regularly as appropriate, which means
it may be less of a roadmap than the traditional kind.

2023-06-07 18:02:30

- reply

📅 Original date posted:2017-06-06
📝 Original message:One thing about BIP148 activation that may be affected by this is the
fact that segwit signalling non-BIP148 miners + BIP148 miners may hold
majority hash power and prevent a chain split. With this SF, that will
no longer be the case, right? Or am I completely confused on the
subject?
On Wed, Jun 7, 2017 at 9:56 AM, James Hilliard via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> Due to the proposed calendar(https://segwit2x.github.io/) for the
> SegWit2x agreement being too slow to activate SegWit mandatory
> signalling ahead of BIP148 using BIP91 I would like to propose another
> option that miners can use to prevent a chain split ahead of the Aug
> 1st BIP148 activation date.
>
> The splitprotection soft fork is essentially BIP91 but using BIP8
> instead of BIP9 with a lower activation threshold and immediate
> mandatory signalling lock-in. This allows for a majority of miners to
> activate mandatory SegWit signalling and prevent a potential chain
> split ahead of BIP148 activation.
>
> This BIP allows for miners to respond to market forces quickly ahead
> of BIP148 activation by signalling for splitprotection. Any miners
> already running BIP148 should be encouraged to use splitprotection.
>
> <pre>
> BIP: splitprotection
> Layer: Consensus (soft fork)
> Title: User Activated Soft Fork Split Protection
> Author: James Hilliard <james.hilliard1 at gmail.com>
> Comments-Summary: No comments yet.
> Comments-URI:
> Status: Draft
> Type: Standards Track
> Created: 2017-05-22
> License: BSD-3-Clause
> CC0-1.0
> </pre>
>
> ==Abstract==
>
> This document specifies a coordination mechanism for a simple majority
> of miners to prevent a chain split ahead of BIP148 activation.
>
> ==Definitions==
>
> "existing segwit deployment" refer to the BIP9 "segwit" deployment
> using bit 1, between November 15th 2016 and November 15th 2017 to
> activate BIP141, BIP143 and BIP147.
>
> ==Motivation==
>
> The biggest risk of BIP148 is an extended chain split, this BIP
> provides a way for a simple majority of miners to eliminate that risk.
>
> This BIP provides a way for a simple majority of miners to coordinate
> activation of the existing segwit deployment with less than 95%
> hashpower before BIP148 activation. Due to time constraints unless
> immediately deployed BIP91 will likely not be able to enforce
> mandatory signalling of segwit before the Aug 1st activation of
> BIP148. This BIP provides a method for rapid miner activation of
> SegWit mandatory signalling ahead of the BIP148 activation date. Since
> the primary goal of this BIP is to reduce the chance of an extended
> chain split as much as possible we activate using a simple miner
> majority of 65% over a 504 block interval rather than a higher
> percentage. This BIP also allows miners to signal their intention to
> run BIP148 in order to prevent a chain split.
>
> ==Specification==
>
> While this BIP is active, all blocks must set the nVersion header top
> 3 bits to 001 together with bit field (1<<1) (according to the
> existing segwit deployment). Blocks that do not signal as required
> will be rejected.
>
> ==Deployment==
>
> This BIP will be deployed by "version bits" with a 65%(this can be
> adjusted if desired) activation threshold BIP9 with the name
> "splitprotecion" and using bit 2.
>
> This BIP starts immediately and is a BIP8 style soft fork since
> mandatory signalling will start on midnight August 1st 2017 (epoch
> time 1501545600) regardless of whether or not this BIP has reached its
> own signalling threshold. This BIP will cease to be active when segwit
> is locked-in.
>
> === Reference implementation ===
>
> <pre>
> // Check if Segregated Witness is Locked In
> bool IsWitnessLockedIn(const CBlockIndex* pindexPrev, const
> Consensus::Params& params)
> {
> LOCK(cs_main);
> return (VersionBitsState(pindexPrev, params,
> Consensus::DEPLOYMENT_SEGWIT, versionbitscache) ==
> THRESHOLD_LOCKED_IN);
> }
>
> // SPLITPROTECTION mandatory segwit signalling.
> if ( VersionBitsState(pindex->pprev, chainparams.GetConsensus(),
> Consensus::DEPLOYMENT_SPLITPROTECTION, versionbitscache) ==
> THRESHOLD_LOCKED_IN &&
> !IsWitnessLockedIn(pindex->pprev, chainparams.GetConsensus()) &&
> // Segwit is not locked in
> !IsWitnessEnabled(pindex->pprev, chainparams.GetConsensus()) ) //
> and is not active.
> {
> bool fVersionBits = (pindex->nVersion & VERSIONBITS_TOP_MASK) ==
> VERSIONBITS_TOP_BITS;
> bool fSegbit = (pindex->nVersion &
> VersionBitsMask(chainparams.GetConsensus(),
> Consensus::DEPLOYMENT_SEGWIT)) != 0;
> if (!(fVersionBits && fSegbit)) {
> return state.DoS(0, error("ConnectBlock(): relayed block must
> signal for segwit, please upgrade"), REJECT_INVALID, "bad-no-segwit");
> }
> }
>
> // BIP148 mandatory segwit signalling.
> int64_t nMedianTimePast = pindex->GetMedianTimePast();
> if ( (nMedianTimePast >= 1501545600) && // Tue 01 Aug 2017 00:00:00 UTC
> (nMedianTimePast <= 1510704000) && // Wed 15 Nov 2017 00:00:00 UTC
> (!IsWitnessLockedIn(pindex->pprev, chainparams.GetConsensus()) &&
> // Segwit is not locked in
> !IsWitnessEnabled(pindex->pprev, chainparams.GetConsensus())) )
> // and is not active.
> {
> bool fVersionBits = (pindex->nVersion & VERSIONBITS_TOP_MASK) ==
> VERSIONBITS_TOP_BITS;
> bool fSegbit = (pindex->nVersion &
> VersionBitsMask(chainparams.GetConsensus(),
> Consensus::DEPLOYMENT_SEGWIT)) != 0;
> if (!(fVersionBits && fSegbit)) {
> return state.DoS(0, error("ConnectBlock(): relayed block must
> signal for segwit, please upgrade"), REJECT_INVALID, "bad-no-segwit");
> }
> }
> </pre>
>
> https://github.com/bitcoin/bitcoin/compare/0.14...jameshilliard:splitprotection-v0.14.1
>
> ==Backwards Compatibility==
>
> This deployment is compatible with the existing "segwit" bit 1
> deployment scheduled between midnight November 15th, 2016 and midnight
> November 15th, 2017. This deployment is also compatible with the
> existing BIP148 deployment. This BIP is compatible with BIP91 only if
> BIP91 activates before it and before BIP148. Miners will need to
> upgrade their nodes to support splitprotection otherwise they may
> build on top of an invalid block. While this bip is active users
> should either upgrade to splitprotection or wait for additional
> confirmations when accepting payments.
>
> ==Rationale==
>
> Historically we have used IsSuperMajority() to activate soft forks
> such as BIP66 which has a mandatory signalling requirement for miners
> once activated, this ensures that miners are aware of new rules being
> enforced. This technique can be leveraged to lower the signalling
> threshold of a soft fork while it is in the process of being deployed
> in a backwards compatible way. We also use a BIP8 style timeout to
> ensure that this BIP is compatible with BIP148 and that BIP148
> compatible mandatory signalling activates regardless of miner
> signalling levels.
>
> By orphaning non-signalling blocks during the BIP9 bit 1 "segwit"
> deployment, this BIP can cause the existing "segwit" deployment to
> activate without needing to release a new deployment. As we approach
> BIP148 activation it may be desirable for a majority of miners to have
> a method that will ensure that there is no chain split.
>
> ==References==
>
> *[https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013714.html
> Mailing list discussion]
> *[https://github.com/bitcoin/bitcoin/blob/v0.6.0/src/main.cpp#L1281-L1283
> P2SH flag day activation]
> *[[bip-0009.mediawiki|BIP9 Version bits with timeout and delay]]
> *[[bip-0016.mediawiki|BIP16 Pay to Script Hash]]
> *[[bip-0091.mediawiki|BIP91 Reduced threshold Segwit MASF]]
> *[[bip-0141.mediawiki|BIP141 Segregated Witness (Consensus layer)]]
> *[[bip-0143.mediawiki|BIP143 Transaction Signature Verification for
> Version 0 Witness Program]]
> *[[bip-0147.mediawiki|BIP147 Dealing with dummy stack element malleability]]
> *[[bip-0148.mediawiki|BIP148 Mandatory activation of segwit deployment]]
> *[[bip-0149.mediawiki|BIP149 Segregated Witness (second deployment)]]
> *[https://bitcoincore.org/en/2016/01/26/segwit-benefits/ Segwit benefits]
>
> ==Copyright==
>
> This document is dual licensed as BSD 3-clause, and Creative Commons
> CC0 1.0 Universal.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

2023-06-07 18:02:15

- reply

📅 Original date posted:2017-06-04
📝 Original message:On Sat, Jun 3, 2017 at 2:55 AM, Alex Akselrod via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> Without a soft fork, this is the only way for light clients to verify that
> peers aren't lying to them. Clients can request headers (just hashes of the
> filters and the previous headers, creating a chain) and look for conflicts
> between peers. If a conflict is found at a certain block, the client can
> download the block, generate a filter, calculate the header by hashing
> together the previous header and the generated filter, and banning any peers
> that don't match. A full node could prune old filters if you wanted and
> recalculate them as necessary if you just keep the filter header chain info
> as really old filters are unlikely to be requested by correctly written
> software but you can't guarantee every client will follow best practices
> either.
Ahh, so you actually make a separate digest chain with prev hashes and
everything. Once/if committed digests are soft forked in, it seems a
bit overkill but maybe it's worth it. (I was always assuming committed
digests in coinbase would come after people started using this, and
that people could just ask a couple of random peers for the digest
hash and ensure everyone gave the same answer as the hash of the
downloaded digest..).
> The simulations are based on completely random data within given parameters.
I noticed an increase in FP hits when using real data sampled from
real scriptPubKeys and such. Address reuse and other weird stuff. See
"lies.h" in github repo for experiments and chainsim.c initial part of
main where wallets get random stuff from the chain.
> I will definitely try to reproduce my experiments with Golomb-Coded
> sets and see what I come up with. It seems like you've got a little
> less than half the size of my digests for 1-block digests but I
> haven't tried making digests for all blocks (and lots of early blocks
> are empty).
>
>
> Filters for empty blocks only take a few bytes and sometimes zero when the
> coinbase output is a burn that doesn't push any data (example will be in the
> test vectors that I'll have ready shortly).
I created digests for all blocks up until block #469805 and actually
ended up with 5.8 GB, which is 1.1 GB lower than what you have, but
may be worse perf-wise on false positive rates and such.
> How fast are these to create? Would it make sense to provide digests
> on demand in some cases, rather than keeping them around indefinitely?
>
>
> They're pretty fast and can be pruned if desired, as mentioned above, as
> long as the header chain is kept.
For comparison, creating the digests above (469805 of them) took
roughly 30 mins on my end, but using the kstats format so probably
higher on an actual node (should get around to profiling that...).

2023-06-07 18:02:08

- reply

📅 Original date posted:2017-06-02
📝 Original message:Hello,
Really wish I'd known you were working on this a few weeks ago, but
such is life. Hopefully I can provide some useful feedback.
On Fri, Jun 2, 2017 at 4:01 AM, Olaoluwa Osuntokun via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> Full-nodes
> maintain an additional index of the chain, and serve this compact filter
> (the index) to light clients which request them. Light clients then fetch
> these filters, query the locally and _maybe_ fetch the block if a relevant
> item matches.
Is it necessary to maintain the index all the way to the beginning of
the chain? When would clients request "really old digests" and why?
> One specific area we'd like feedback on is the parameter selection. Unlike
> BIP-37 which allows clients to dynamically tune their false positive rate,
> our proposal uses a _fixed_ false-positive. Within the document, it's
> currently specified as P = 1/2^20. We've done a bit of analysis and
> optimization attempting to optimize the following sum:
> filter_download_bandwidth + expected_block_false_positive_bandwidth. Alex
> has made a JS calculator that allows y'all to explore the affect of
> tweaking the false positive rate in addition to the following variables:
> the number of items the wallet is scanning for, the size of the blocks,
> number of blocks fetched, and the size of the filters themselves. The
> calculator calculates the expected bandwidth utilization using the CDF of
> the Geometric Distribution. The calculator can be found here:
> https://aakselrod.github.io/gcs_calc.html. Alex also has an empirical
> script he's been running on actual data, and the results seem to match up
> rather nicely.
I haven't tried the tool yet, and maybe it will answer some of my questions.
On what data were the simulated wallets on actual data based? How did
false positive rates for wallets with lots of items (pubkeys etc) play
out? Is there a maximum number of items for a wallet before it becomes
too bandwidth costly to use digests?
> We we're excited to see that Karl Johan Alm (kallewoof) has done some
> (rather extensive!) analysis of his own, focusing on a distinct encoding
> type [5]. I haven't had the time yet to dig into his report yet, but I
> think I've read enough to extract the key difference in our encodings: his
> filters use a binomial encoding _directly_ on the filter contents, will we
> instead create a Golomb-Coded set with the contents being _hashes_ (we use
> siphash) of the filter items.
I will definitely try to reproduce my experiments with Golomb-Coded
sets and see what I come up with. It seems like you've got a little
less than half the size of my digests for 1-block digests but I
haven't tried making digests for all blocks (and lots of early blocks
are empty).
On the BIP proposal itself:
In Compact Filter Header Chain, you mention that clients should
download filters from nodes if filter_headers is not identical, and
ban offending nodes. What about temporary forks in the chain? What
about longer forks? In general, I am curious how you will deal with
reorgs and temporary non-consensus related chain splits.
I am also curious if you have considered digests containing multiple
blocks. Retaining a permanent binsearchable record of the entire chain
is obviously too space costly, but keeping the last X blocks as
binsearchable could speed up syncing for clients tremendously, I feel.
It may also be space efficient to ONLY store older digests in chunks
of e.g. 8 blocks. A client syncing up finding a match in an 8-block
chunk would have to grab those 8 blocks, but if it's not recent, that
may be acceptable. It may even be possible to make 4-, 2-, 1-block
digests on demand.
How fast are these to create? Would it make sense to provide digests
on demand in some cases, rather than keeping them around indefinitely?

2023-06-07 18:02:08

- reply

2023-06-07 18:02:02

- reply

📅 Original date posted:2017-05-31
📝 Original message:Hello,
I have spent a fair bit of time trying to nail how exactly block
filter digests[1] should be done to optimize bandwidth, space,
resource usage.
The report can be found here: http://bc-2.jp/bfd-profile.pdf
This graph shows bandwidth use of 200 wallets simulated over 5000
blocks: http://bc-2.jp/bandwidth_bfd.png (black line is "sync once per
block" wallet, yellow is "sync once per 144 blocks" wallet, red is
average across all wallets).
An interesting insight made during the experiments: when allowing
digests to contain multiple blocks, the false positive rate of high
block count digests can be higher than normal, because the probability
of a false positive hit for a given entry in multiple digests,
assuming their sizes differ, is almost completely independent.
The results look rather promising to me, but I would like to hear
comments, in particular on the approach taken, if I made any faulty
assumptions, bad math mistakes, etc.
I am also curious what people consider to be acceptable costs in terms
of bandwidth use and memory (I couldn't find any stats on bandwidth
use of bloom filters). In the profiling, I restricted the field sizes
to 2^27 = 128 MB. I assumed this was appropriate as these fields are
very short lived, and in worst case, a client *could* do the scan and
decode simultaneously, without allocating up the space for the field
at all. For high block count digests (e.g. 1024 blocks), this is
sometimes overfilled. I wonder if 2^28 (256 MB) fields would be at all
acceptable or if an over-filled (high false positive rate) field is
better.
For that matter, I am not entirely sure 1024-block digests are
necessary, but they do come with an average 15 kb/block which is
pretty good.
I also wonder if the serialization approach taken is overkill or not.
It does save some space instead of simply storing "BBBAAAAA" but adds
some complexity that may not be warranted.
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-May/012636.html

2023-06-07 18:00:54

- reply

📅 Original date posted:2017-05-07
📝 Original message:Hello,
I am proposing a new feature for rate limiting purposes where nodes
can make and solve arbitrary PoW challenges in return for connection
slots (to be expanded to cover e.g. bloom filters or other DoS risky
services).
The BIP currently includes two proofs of work (sha256 and
cuckoo-cycle) which can be combined (e.g. sha256(cuckoo-cycle) or
sha256(sha256(sha256)), etc).
Link: https://github.com/kallewoof/bips/blob/pow-connection-slots/bip-rate-limiting-via-pow.mediawiki
Feedback welcome.

2023-06-07 18:00:54

- reply

📅 Original date posted:2017-05-08
📝 Original message:Erik,
On Tue, May 9, 2017 at 3:58 AM, Erik Aronesty <erik at q32.com> wrote:
> - It would be cool if any rate-limiting POW was specified as bytecode ... so
> nodes can plug in as many "machine-captcha" things as they please, and
> solvers can choose to solve... or just say "nope too hard".
I'm not entirely sure what you mean, but right now you can make an
arbitrary chain of challenges, and the BIP includes methods for
determining an approximate time to solve (nodes will, at the very
least, discard any challenge which will on average take longer time to
solve than the expiration of the challenge itself, for example, i.e.
the "nope too hard" part).
> - Alternately, it would be a lot nicer if you just required people to pay a
> nanobit .... that could prevent DDOS even better, and generate a revenue
> stream for nodes.
Others mentioned this approach. I haven't given it much thought.
Admittedly it would be an effective way to prevent DoS but it also has
some unwanted side effects that need to be cleared up (e.g. in a
no-gains scenario like the BIP proposes, the node requesting PoW done
doesn't *gain* anything from lying to the node performing the work).

2023-06-07 18:00:54

- reply

📅 Original date posted:2017-05-19
📝 Original message:Hello,
Some time has passed since this was initially posted, and I have not
received any negative feedback. If no objections are raised, I would
like to have a BIP number assigned.
On Mon, May 8, 2017 at 11:48 AM, Karl Johan Alm
<karljohan-alm at garage.co.jp> wrote:
> Hello,
>
> I am proposing a new feature for rate limiting purposes where nodes
> can make and solve arbitrary PoW challenges in return for connection
> slots (to be expanded to cover e.g. bloom filters or other DoS risky
> services).
>
> The BIP currently includes two proofs of work (sha256 and
> cuckoo-cycle) which can be combined (e.g. sha256(cuckoo-cycle) or
> sha256(sha256(sha256)), etc).
>
> Link: https://github.com/kallewoof/bips/blob/pow-connection-slots/bip-rate-limiting-via-pow.mediawiki
>
> Feedback welcome.

2023-06-07 18:00:40

- reply

📅 Original date posted:2017-05-23
📝 Original message:On Tue, May 23, 2017 at 1:03 PM, Steven Pine via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> Correct me if I am wrong, but currently core developers are arguing over
> whether or not to allow an optional configuration switch which defaults off
> but signals and enforces BIP148 when used. Who are we protecting users from,
> themselves? Are you protecting core? from what? I am somewhat genuinely
> befuddled by those who can't even allow a user config switch to be set.
Essentially, if we make a potentially very harmful option easy to
enable for users, we are putting them at risk, so yes, this is about
protecting users of the base Bitcoin Core implementation. Users have,
hopefully, come to appreciate this implementation for the peer
review-based strict development process, and making a hasty decision
due to time constraints (segwit activation expiration) may have
undesirable consequences. Opinions among the regular contributors are
split on the matter, which to me is an indication we should be
cautious and consider all aspects before making a decision on the
matter.

2023-06-07 18:00:40

- reply