Dave Hudson [ARCHIVE] / is on Nostr

⚠️ This account is not associated with a real person. It's an archival account for mailing list messages.
🗓️ Active from 2015-05-07 to 2016-03-09
💬 Posted a total of 14 messages
📚 Participated in the following categories: Bitcoin-development, bitcoin-dev
🔇 Please do not DM this user as it's used for archive purposes only.

Last Notes

2023-06-07 17:49:33

- reply

📅 Original date posted:2016-03-09
📝 Original message:> On 9 Mar 2016, at 20:21, Bob McElrath <bob_bitcoin at mcelrath.org> wrote:
>
> Dave Hudson [dave at hashingit.com] wrote:
>> A damping-based design would seem like the obvious choice (I can think of a
>> few variations on a theme here, but most are found in the realms of control
>> theory somewhere). The problem, though, is working working out a timeframe
>> over which to run the derivative calculations.
>
> From a measurement theory perspective this is straightforward. Each block is a
> measurement, and error propagation can be performed to derive an error on the
> derivatives.
Sure, but I think there are 2 problems:
1) My guess is that errors over anything but a long period are probably too large to be very useful.
2) We don't have a strong notion of time that is part of the consensus. Sure, blocks have timestamps but they're very loosely controlled (can't be more than 2 hours ahead of what any validating node thinks the time might be). Difficulty can't be calculated based on anything that's not part of the consensus data.
> The statistical theory of Bitcoin's block timing is known as a Poisson Point
> Process: https://en.wikipedia.org/wiki/Poisson_point_process or temporal point
> process. If you google those plus "estimation" you'll find a metric shit-ton of
> literature on how to handle this.
Strictly it's a non-homogeneous Poisson Process, but I'm pretty familiar with the concept (Google threw one of my own blog posts back at me: http://hashingit.com/analysis/27-hash-rate-headaches, but I actually prefer this one: http://hashingit.com/analysis/30-finding-2016-blocks because most people seem to find it easier to visualize).
>> The problem is the measurement of the hashrate, which is pretty inaccurate at
>> best because even 2016 events isn't really enough (with a completely constant
>> hash rate running indefinitely we'd see difficulty swings of up to +/- 5% even
>> with the current algorithm). In order to meaningfully react to a major loss
>> of hashing we'd still need to be considering a window of probably 2 weeks.
>
> You don't want to assume it's constant in order to get a better measurement.
> The assumption is clearly false. But, errors can be calculated, and retargeting
> can take errors into account, because no matter what we'll always be dealing
> with a finite sample.
Agreed, it's a thought experiment I ran in May 2014 (http://hashingit.com/analysis/28-reach-for-the-ear-defenders). I found that many people's intuition is that there would be little or no difficulty changes in such a scenario, but the intuition isn't reliable. Given a static hash rate the NHPP behaviour introduces a surprisingly large amount of noise (often much larger than any signal over a period of even weeks). Any measurements in the order of even a few days has so much noise that it's practically unusable. I just realized that unlike some of my other sims this one didn't make it to github; I'll fix that later this week.
Cheers,
Dave

2023-06-07 17:49:33

- reply

📅 Original date posted:2016-03-09
📝 Original message:A damping-based design would seem like the obvious choice (I can think of a few variations on a theme here, but most are found in the realms of control theory somewhere). The problem, though, is working working out a timeframe over which to run the derivative calculations.
The problem is the measurement of the hashrate, which is pretty inaccurate at best because even 2016 events isn't really enough (with a completely constant hash rate running indefinitely we'd see difficulty swings of up to +/- 5% even with the current algorithm). In order to meaningfully react to a major loss of hashing we'd still need to be considering a window of probably 2 weeks.
My other concern is that if we allow quick retargets to lower difficulties then that seems likely to expose the chain to being gamed. I'd need to think about this some more, but a few scenarios I was thinking about earlier this week appeared to risk making some types of selfish mining strategies quite a lot more profitable.
With all this said though I'll be very surprised if there's a huge drop in the hash rate come July. The hash rate has jumped up by almost 70% in the last 6 to 7 months and that implies some pretty serious investments by miners who are quite aware of the halving. My guess is that quite a lot of the baseline 30% has also been replaced in the same cycle. These same miners were mining with a coin price around $250 last year so in terms of profitability I'm pretty sure that one around $400 won't be a huge concern.
I'm sure that there will be some very public "I'm done with mining" announcements from a few smaller miners come July, but I suspect the bulk of the network will have a relatively small blip and continue on its way.
Cheers,
Dave
> On 8 Mar 2016, at 22:05, Bob McElrath <bob_bitcoin at mcelrath.org> wrote:
>
> Dave Hudson via bitcoin-dev [bitcoin-dev at lists.linuxfoundation.org] wrote:
>> I think the biggest question here would be how would the difficulty
>> retargeting be changed? Without seeing the algorithm proposal it's difficult
>> to assess the impact that it would have, but my intuition is that this is
>> likely to be problematic.
>
> I have no comment on whether this will be *needed* but there's a simple
> algorithm that I haven't seen any coin adopt, that I think needs to be: the
> critically damped harmonic oscillator:
>
> http://mathworld.wolfram.com/CriticallyDampedSimpleHarmonicMotion.html
>
> In dynamical systems one does a derivative expansion. Here we want to find the
> first and second derivatives (in time) of the hashrate. These can be determined
> by a method of finite differences, or fancier algorithms which use a quadratic
> or quartic polynomial approximation. Two derivatives are generally all that is
> needed, and the resulting dynamical system is a damped harmonic oscillator.
>
> A damped harmonic oscillator is basically how your car's shock absorbers work.
> The relevant differential equation has two parameters: the oscillation frequency
> and damping factor. The maximum oscillation frequency is the block rate. Any
> oscillation faster than the block rate cannot be measured by block times. The
> damping rate is an exponential decay and for critical damping is twice the
> oscillation frequency.
>
> So, this is a zero parameter, optimal damping solution for a varying hashrate.
> This is inherently a numeric approximation solution to a differential equation,
> so questions of approximations for the hashrate enter, but that's all. Weak
> block proposals will be able to get better approximations to the hashrate.
>
> If solving this problem is deemed desirable, I can put some time into this, or
> direct others as to how to go about it.
>
> --
> Cheers, Bob McElrath
>
> "For every complex problem, there is a solution that is simple, neat, and wrong."
> -- H. L. Mencken
>

2023-06-07 17:49:32

- reply

📅 Original date posted:2016-03-02
📝 Original message:I think the biggest question here would be how would the difficulty retargeting be changed? Without seeing the algorithm proposal it's difficult to assess the impact that it would have, but my intuition is that this is likely to be problematic.
Probabilistically the network sees surprisingly frequent swings of +/-20% in terms of the block finding rate on any given day, while the statistical noise over a 2016 block period can be more than +/-5%. Any change would still have to require a fairly significant period of time before there would be a reasonable level of confidence that the hash rate really had fallen as opposed to just seeing statistical noise (http://hashingit.com/analysis/29-lies-damned-lies-and-bitcoin-difficulties and http://hashingit.com/analysis/28-reach-for-the-ear-defenders).
How long would be required to deem that the hash rate had dramatically fallen? Would such a change be a one-time event or would it be ever-present?
If we were to say that if the hash rate dropped 50% in one day (which could, of course be a 30% real drop and 20% variance) and the difficulty was retargeted to 50% lower then that would have to be matched with a similar rapid retarget if it were to increase by a similar amount. Failing to do this both ways this would introduce an economic incentive for large miners to suppress the difficulty and gain dramatically larger numbers of block rewards. The current fixed block count per difficulty change prevents this because the daily losses while suppressing hashing outweigh the potential gains when it's re-added.
Cheers,
Dave
> On 2 Mar 2016, at 14:56, Luke Dashjr via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> We are coming up on the subsidy halving this July, and there have been some
> concerns raised that a non-trivial number of miners could potentially drop off
> the network. This would result in a significantly longer block interval, which
> also means a higher per-block transaction volume, which could cause the block
> size limit to legitimately be hit much sooner than expected. Furthermore, due
> to difficulty adjustment being measured exclusively in blocks, the time until
> it adjusts to compensate would be prolonged.
>
> For example, if 50% of miners dropped off the network, blocks would be every
> 20 minutes on average and contain double the transactions they presently do.
> Even double would be approximately 850-900k, which potentially bumps up
> against the hard limit when empty blocks are taken into consideration. This
> situation would continue for a full month if no changes are made. If more
> miners drop off the network, most of this becomes linearly worse, but due to
> hitting the block size limit, the backlog would grow indefinitely until the
> adjustment occurs.
>
> To alleviate this risk, it seems reasonable to propose a hardfork to the
> difficulty adjustment algorithm so it can adapt quicker to such a significant
> drop in mining rate. BtcDrak tells me he has well-tested code for this in his
> altcoin, which has seen some roller-coaster hashrates, so it may even be
> possible to have such a proposal ready in time to be deployed alongside SegWit
> to take effect in time for the upcoming subsidy halving. If this slips, I
> think it may be reasonable to push for at least code-readiness before July,
> and possibly roll it into any other hardfork proposed before or around that
> time.
>
> I am unaware of any reason this would be controversial, so if anyone has a
> problem with such a change, please speak up sooner rather than later. Other
> ideas or concerns are of course welcome as well.
>
> Thanks,
>
> Luke
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

2023-06-07 17:33:36

- reply

📅 Original date posted:2015-08-07
📝 Original message:> On 7 Aug 2015, at 16:17, Ryan Butler via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> A raspberry pie 2 node on reasonable Internet connection with a reasonable hard drive can run a node with 8 or 20mb blocks easily.
>
I'm curious as I've not seen any data on this subject. How fast can a RP2 do the necessary cryptographic calculations to validate blocks of various sizes?
While everyone tends to talk in terms of 10 minutes per block that is, of course, only a typical time and doesn't account for situations in which 2 or more blocks are found in quick succession (which, of course, happens on a daily basis). At what point does, say, an RP2 node fail to be able to validate a second or third block because it's still not finished processing the first?
If someone were to be playing games with the system and mining transactions without first broadcasting them to the network then how long would that take? This would in essence define the ability to DoS lower-performance nodes (ignoring all of the other usual considerations such as bandwidth, etc).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150807/80bcb76d/attachment-0001.html>

2023-06-07 17:32:50

- reply

📅 Original date posted:2015-08-05
📝 Original message:>
> On 5 Aug 2015, at 15:15, Peter R <peter_r at gmx.com> wrote:
>
> Hi Dave,
>
> Thank you for the feedback regarding my paper.
>
>> The paper is nicely done, but I'm concerned that there's a real problem with equation 4. The orphan rate is not just a function of time; it's also a function of the block maker's proportion of the network hash rate. Fundamentally a block maker (pool or aggregation of pools) does not orphan its own blocks.
>
> With the benefit of hindsight, I think the paper would be stronger if it also analyzed how the model changes (or doesn't) if we assume zero propagation impedance for intra-miner communication, as you suggested (the "you don't orphan your own blocks" idea). Note that the paper did briefly discuss miner-dependent propagation times in the second paragraph of page 9 and in note 13.
I think this would be really interesting. It's an area that seems to be lacking research.
While I've not had time to model it I did have a quick discussion with the author of the Organ-of-Corti blog a few months ago and he seemed to think that the Poisson process model isn't quite accurate here. Intuitively this makes sense as until a block has fully propagated we're only seeing some fraction of the actual hashing network operating on the same problem, so we actually see slightly fewer very quick blocks than we might expect.
I do suspect that if we were to model this more accurately we might be able to infer the "typical" propagation characteristics by measuring the deviation from the expected distribution.
>> Consider that a 1% miner must assume a greater risk from orphaning than, say, a pool with 25%, or worse 40% of the hash rate.
>
> I'd like to explore this in more detail. Although a miner may not orphan his own block, by building on his own block he may now orphan two blocks in a row. At some point, his solution or solutions must be communicated to his peers. And if there's information about the transactions in his blocks to communicate, I think there's a cost associated with that. It's an interesting problem and I'd like to continue working on it. \
Agreed - I think this would be really interesting!
>> I suspect this may well change some of the conclusions as larger block makers will definitely be able to create larger blocks than their smaller counterparts.
>
> It will be interesting to see. I suspect that the main result that "a healthy fee market exists" will still hold (assuming of course that a single miner with >50% of the hash power isn't acting maliciously). Whether miners with larger value of h/H have a profit advantage, I'm not sure (but that was outside the scope of the paper anyways).
I really look forward to seeing the revised version. Seeing the differences will also help assess how much impact there is from simplified models.
Regards,
Dave

2023-06-07 17:32:49

- reply

📅 Original date posted:2015-08-04
📝 Original message:> On 4 Aug 2015, at 14:30, Gavin Andresen <gavinandresen at gmail.com> wrote:
>
> On Tue, Aug 4, 2015 at 2:41 PM, Dave Hudson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
> Fundamentally a block maker (pool or aggregation of pools) does not orphan its own blocks.
>
> Unless the block maker has an infinitely fast connection to it's hashpower OR it's hashpower is not parallelized at all, that's not strictly true -- it WILL orphan its own blocks because two hashing units will find solutions in the time it takes to communicate that solution to the block maker and to the rest of the hashing units.
>
> That's getting into "how many miners can dance on the head of a pin" territory, though. I don't think we know whether the communication advantages of putting lots of hashing power physically close together will outweigh the extra cooling costs of doing that (or maybe some other tradeoff I haven't thought of). That would be a fine topic for another paper....
Yes, but the block maker won't publish the second block it finds for the same set of transactions. It won't orphan its own block. In fact even if it does it still doesn't matter because the block maker still gets the block reward irrespective of which of the two solutions are published.
It's not about which hash wins, the issue is who gets paid as a result.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150804/f8ac9aae/attachment.html>

2023-06-07 17:32:47

- reply

📅 Original date posted:2015-08-04
📝 Original message:The paper is nicely done, but I'm concerned that there's a real problem with equation 4. The orphan rate is not just a function of time; it's also a function of the block maker's proportion of the network hash rate. Fundamentally a block maker (pool or aggregation of pools) does not orphan its own blocks. In a degenerate case a 100% pool has no orphaned blocks. Consider that a 1% miner must assume a greater risk from orphaning than, say, a pool with 25%, or worse 40% of the hash rate.
I suspect this may well change some of the conclusions as larger block makers will definitely be able to create larger blocks than their smaller counterparts.
Cheers,
Dave
> On 3 Aug 2015, at 23:40, Peter R via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Dear Bitcoin-Dev Mailing list,
>
> I’d like to share a research paper I’ve recently completed titled “A Transaction Fee Market Exists Without a Block Size Limit.” In addition to presenting some useful charts such as the cost to produce large spam blocks, I think the paper convincingly demonstrates that, due to the orphaning cost, a block size limit is not necessary to ensure a functioning fee market.
>
> The paper does not argue that a block size limit is unnecessary in general, and in fact brings up questions related to mining cartels and the size of the UTXO set.
>
> It can be downloaded in PDF format here:
>
> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf <https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf>
>
> Or viewed with a web-browser here:
>
> https://www.scribd.com/doc/273443462/A-Transaction-Fee-Market-Exists-Without-a-Block-Size-Limit <https://www.scribd.com/doc/273443462/A-Transaction-Fee-Market-Exists-Without-a-Block-Size-Limit>
>
> Abstract. This paper shows how a rational Bitcoin miner should select transactions from his node’s mempool, when creating a new block, in order to maximize his profit in the absence of a block size limit. To show this, the paper introduces the block space supply curve and the mempool demand curve. The former describes the cost for a miner to supply block space by accounting for orphaning risk. The latter represents the fees offered by the transactions in mempool, and is expressed versus the minimum block size required to claim a given portion of the fees. The paper explains how the supply and demand curves from classical economics are related to the derivatives of these two curves, and proves that producing the quantity of block space indicated by their intersection point maximizes the miner’s profit. The paper then shows that an unhealthy fee market—where miners are incentivized to produce arbitrarily large blocks—cannot exist since it requires communicating information at an arbitrarily fast rate. The paper concludes by considering the conditions under which a rational miner would produce big, small or empty blocks, and by estimating the cost of a spam attack.
>
> Best regards,
> Peter
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150804/793ae4f2/attachment.html>

- reply

- reply

- reply

- reply

- reply

📅 Original date posted:2015-07-30
📝 Original message:> On 30 Jul 2015, at 06:14, Tom Harding via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Another empirical fact also needs explaining. Why have average fees *as
> measured in BTC* risen during the times of highest public interest in
> bitcoin? This happened without block size pressure, and it is not an
> exchange rate effect -- these are raw BTC fees:
>
> https://blockchain.info/charts/transaction-fees?timespan=all&daysAverageString=7 <https://blockchain.info/charts/transaction-fees?timespan=all&daysAverageString=7>
I've not published any new figures for about 8 months (will try to do that this weekend), but the thing that that chart doesn't show is what's actually happening to fees per transaction. Here's a chart that does: http://hashingit.com/analysis/35-the-future-of-bitcoin-transaction-fees <http://hashingit.com/analysis/35-the-future-of-bitcoin-transaction-fees>
The data is also taken from blockchain.info so it's apples-for-apples. It shows that far from a fees going up they spent 3 years dropping. I just ran a new chart and the decline in fees continued until about 8 weeks when the "stress tests" first occurred. Even so, they're still below the level from the end of 2013. By comparison the total transaction volume is up about 2.4x to 2.5x (don't have the exact number).
> ... more evidence that conclusively refutes the conjecture that a
> production quota is necessary for a "functioning fee market." A
> production quota merely pushes up fees. We have a functioning market,
> and so far, it shows that wider bitcoin usage is even more effective
> than a quota at pushing up fees.
I think it's equally easy to argue (from the same data) that wider adoption has actually caused wallet users to become much more effective at fee selection. Miners (as expected, assuming that they hadn't formed a cartel) have continued to accept whatever fees are available, no matter how small. Only where there has been an element of scarcity have we actually seen miners do anything but take whatever is offered.
Clearly history is not an accurate indicator of what might happen in the future, but it seems difficult to argue that there has been any sort of fee market emerge to date (other than as a result of scarcity during the stress tests).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150730/aaba0a9a/attachment-0001.html>

2023-06-07 15:41:57

- reply

📅 Original date posted:2015-07-11
📝 Original message:> On 11 Jul 2015, at 14:17, Tier Nolan <tier.nolan at gmail.com> wrote:
>
> On Sat, Jul 11, 2015 at 1:09 PM, Nathan Wilcox <nathan at leastauthority.com <mailto:nathan at leastauthority.com>> wrote:
>
> If there's any cost to non-SPV mining, and the chance that an incoming block contains only valid transactions is very high, isn't there still an incentive to make this timeout longer and longer?
>
> The benefit drops off pretty quickly as the timeout increases (and eventually goes negative).
>
> Hashers would end up getting variable payouts based on how long since the last block was received. If some miners increase/decrease their output based on the fees the pools offer, then as time passes since the last block, the hashing rate would increase. This reduces the variation in block to block times.
>
> For example, if there was 1.5MB of transactions in the memory pool and they all paid the same fee per byte, then the block would be a full block at that rate. However, there would only be 0.5MB of transactions left. This means that the next block would be half full and only be able to pay out 50% of the fee, but the difficulty would be the same. The pay per hash would be 50% lower. Once 0.5MB of new transactions arrive, the fee would be back up to the same as the previous block.
This would probably be worse. The 1 MB would include the highest fee transactions, leaving the lowest fees in the remaining 0.5 MB.
> If there are major SHA256 altcoins (or side chains), then miners might end up switching between coins. The longer a coin went without a new block being found, the more tx fees available in the memory pool, so the more hashing power would decide to switch to that coin.
>
> There could be a situation where adding a few more transactions to the memory pool of a coin would cause a 100X increasing in hashing until the block was found and then it stalling again as the hashing power switches away. This is similar to alt coins getting blasted by coin switching pools and then dropping to almost no power.
If hashing isn't constantly applied all the time then the inter-block times will expand and the difficulty will reduce. This means that a pool that decides to use all of its available hashing 100% of the time now has a distinct advantage over those who are playing nicely. This is the same problem that the "proof of idle" idea had; it only works if no-one chooses to try to exploit it (which seems very unlikely).
One might ask why a pool might wish to try to exploit this, but let's assume we have a fees-only reward, so here's a quick thought experiment - the numbers are only approximate and would need a thorough simulation but serve for discussion:
Say, for the sake of argument that over a nominal 10 minute period we see 10 BTC worth of transaction fees. If the mempool is empty of interesting fees at the start of a block then we might like to imagine that rational miners will power down their hashing to save energy costs until the fees are worthwhile. Let's assume, for the sake of argument, that this nominally takes 5 minutes.
After 5 minutes we go from 0% to 100% as all hashing engines switch on. The difficulty will have corrected to mean that 100% of the work will nominally happen in the next 5 minutes, but that means that a malicious miner has a 2x amplification of their nominal hashing rate to do mischief in the preceding 5 minutes.
Such a malicious miner would choose to spend their 5 minutes re-mining the previous block, but dropping some amount of the transactions from it. Let's say that they try to re-mine only 9.5 BTC out the previous 10 BTC. If they succeed then they're offering everyone else an extra 0.5 BTC (5%) if they mine on top of their re-mined block and as an incentive to orphan the original block. Rational miners would definitely choose to build on the re-mined block because they get more reward from doing so.
Of course the extra hashing that our malicious miner is doing will actually push the difficulty back up somewhat, but they're still running at an advantage over the long-term. I've also ignored some of the other security implications of the hashing amplification effects (e.g. 25% of the hash rate can end up controlling 50% of the blocks in the scenario above).
An obvious objection to this scenario is that everyone would notice the malicious mining. Statistically, yes, the orphan rate would be much higher, but there's no guarantee that anyone could ever work out who was behind this. It's all too easy to create an "unknown" pool, or a series of "unknown" pools. Of course our malicious miner might choose to only target blocks that had particularly high fees associated with it, in which case the orphan rate might barely change.
The only way I can see that this wouldn't be the case would be if there were always useful fees available to mine immediately after a block is found. If block space is kept moderately scarce then immediately a block is found then everyone will keep mining and the incentives to try to deliberately orphan the last block are dramatically reduced.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150711/3208fd52/attachment.html>

2023-06-07 15:36:00

- reply

📅 Original date posted:2015-05-31
📝 Original message:> On 31 May 2015, at 13:52, Gavin Andresen <gavinandresen at gmail.com> wrote:
>
> On Sat, May 30, 2015 at 9:31 PM, Chun Wang <1240902 at gmail.com <mailto:1240902 at gmail.com>> wrote:
> If someone propagate a 20MB block, it will take at best 6 seconds for
> us to receive to verify it at current configuration, result of one
> percent orphan rate increase.
>
> That orphan rate increase will go to whoever is producing the 20MB blocks, NOT you.
There's an interesting incentives question if the mining fees ever become large enough to be interesting. Given two potential blocks on which to build then for the best interests of the system we'd want miners to select the block that confirmed the largest number of transactions since that puts less pressure on the network later. This is at odds with the incentives for our would-be block maker though because the incentive for mining would be to use whichever block left the largest potential fees available; that's generally going to be the smaller of the two.
This, of course, only gets worse as the block reward reduces and fees become the dominant way for miners to be paid (and my hypothesis that eventually this could lead to miners trying to deliberately orphan earlier blocks to "steal" fees because the fixed block reward is no longer the dominant part of their income).
When coupled with the block propagation delay problem increasing the risk of orphan races I'm pretty sure that this actually leads to miners having an incentive to continually mine smaller blocks, and that's aside from the question of whether smaller blocks will push up fees (which also benefits miners).
Cheers,
Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150531/2622fb5b/attachment.html>

2023-06-07 15:34:50

- reply

📅 Original date posted:2015-05-12
📝 Original message:I think proof-of-idle had a potentially serious problem when I last looked at it. The risk is that a largish miner can use everyone else's idle time to construct a very long chain; it's also easy enough for them to make it appear to be the work of a large number of distinct miners. Given that this would allow them to arbitrarily re-mine any block rewards and potentially censor any transactions then that just seems like a huge security hole?
Cheers,
Dave
> On 12 May 2015, at 17:10, Gavin Andresen <gavinandresen at gmail.com> wrote:
>
> Added back the list, I didn't mean to reply privately:
>
> Fair enough, I'll try to find time in the next month or three to write up four plausible future scenarios for how mining incentives might work:
>
> 1) Fee-supported with very large blocks containing lots of tiny-fee transactions
> 2) Proof-of-idle supported (I wish Tadge Dryja would publish his proof-of-idle idea....)
> 3) Fees purely as transaction-spam-prevention measure, chain security via alternative consensus algorithm (in this scenario there is very little mining).
> 4) Fee supported with small blocks containing high-fee transactions moving coins to/from sidechains.
>
> Would that be helpful, or do you have some reason for thinking that we should pick just one and focus all of our efforts on making that one scenario happen?
>
> I always think it is better, when possible, not to "bet on one horse."
>
>
> On Tue, May 12, 2015 at 10:39 AM, Thomas Voegtlin <thomasv at electrum.org <mailto:thomasv at electrum.org>> wrote:
> Le 12/05/2015 15:44, Gavin Andresen a écrit :
> > Ok, here's my scenario:
> >
> > https://blog.bitcoinfoundation.org/a-scalability-roadmap/ <https://blog.bitcoinfoundation.org/a-scalability-roadmap/>
> >
> > It might be wrong. I welcome other people to present their road maps.
> >
>
> [answering to you only because you answered to me and not to the list;
> feel free to repost this to the list though]
>
> Yes, that's exactly the kind of roadmap I am asking for. But your blog
> post does not say anything about long term mining incentives, it only
> talks about scalability. My point is that we need the same kind of thing
> for miners incentives.
>
>
>
> --
> --
> Gavin Andresen
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y_______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150512/73ba86e4/attachment.html>

2023-06-07 15:34:42

- reply

📅 Original date posted:2015-05-11
📝 Original message:I proposed the same thing last year (there's a video of the presentation I was giving somewhere around). My intuition was that this would require slowly reducing the inter-block time, probably by step reductions at particular block heights.
Having had almost a year to think about it some more there are a few subtleties:
1) I think it could discourage decentralisation if the nominal 2 week period per difficulty retarget is retained. If we reached 4032 blocks and a 5 minute block time then there would be 2x as many blocks at any given difficulty which increases the odds of a smaller pool finding a block and thus getting a reward. Block rewards would have to drop in proportion to the reduced interval to keep the total schedule of 21M coins on track though, but the reduction in variance is a win for smaller miners.
2) There are limits to the block time. The speed of light is an ultimately limiting factor here, but we would want to avoid excessive orphan rates.
3) There would be some amount of confusion about numbers of confirmations. I actually think that confirmation numbers are a really misleading idea anyway and it would be safer to think in terms of "minutes of security". A zero conf transaction has "zero minutes", while right now 1, 2, 3 and 6 would be "ten minutes", "twenty minutes", "thirty minutes" and "sixty minutes" respectively. If our block time were 5 minutes then 8 confirmations would be "forty minutes" of security; if the block time was 2.5 minutes then 8 confirmations would be "twenty minutes" of security. The "minutes of security" measure indicates the mean number of minutes of the entire network's hash rate would be required to undo a transaction.
4) Reducing the inter-block time reduces the variance in reaching that "sixty minutes" of security level. The variance around finding 6 blocks with a ten minute interval is much wider than the variance for finding 12 blocks with a 5 minute interval.
> On 11 May 2015, at 08:30, Thy Shizzle <thyshizzle at outlook.com> wrote:
>
> Yes This!
>
> So many people seem hung up on growing the block size! If gaining a higher tps throughput is the main aim, I think that this proposition to speed up block creation has merit!
>
> Yes it will lead to an increase in the block chain still due to 1mb ~1 minute instead of ~10 minute, but the change to the protocol is minor, you are only adding in a different difficulty rate starting from hight blah, no new features or anything are being added so there seems to me much less of a security risk! Also that impact if a hard fork should be minimal because there is nothing but absolute incentive for miners to mine at the new easier difficulty!
>
> I feel this deserves a great deal of consideration as opposed to blowing out the block through miners voting etc!!!!
> From: Sergio Lerner <mailto:sergiolerner at certimix.com>
> Sent: ‎11/‎05/‎2015 5:05 PM
> To: bitcoin-development at lists.sourceforge.net <mailto:bitcoin-development at lists.sourceforge.net>
> Subject: [Bitcoin-development] Reducing the block rate instead of increasing the maximum block size
>
> In this e-mail I'll do my best to argue than if you accept that
> increasing the transactions/second is a good direction to go, then
> increasing the maximum block size is not the best way to do it. I argue
> that the right direction to go is to decrease the block rate to 1
> minute, while keeping the block size limit to 1 Megabyte (or increasing
> it from a lower value such as 100 Kbyte and then have a step function).
> I'm backing up my claims with many hours of research simulating the
> Bitcoin network under different conditions [1]. I'll try to convince
> you by responding to each of the arguments I've heard against it.
>
> Arguments against reducing the block interval
>
> 1. It will encourage centralization, because participants of mining
> pools will loose more money because of excessive initial block template
> latency, which leads to higher stale shares
>
> When a new block is solved, that information needs to propagate
> throughout the Bitcoin network up to the mining pool operator nodes,
> then a new block header candidate is created, and this header must be
> propagated to all the mining pool users, ether by a push or a pull
> model. Generally the mining server pushes new work units to the
> individual miners. If done other way around, the server would need to
> handle a high load of continuous work requests that would be difficult
> to distinguish from a DDoS attack. So if the server pushes new block
> header candidates to clients, then the problem boils down to increasing
> bandwidth of the servers to achieve a tenfold increase in work
> distribution. Or distributing the servers geographically to achieve a
> lower latency. Propagating blocks does not require additional CPU
> resources, so mining pools administrators would need to increase
> moderately their investment in the server infrastructure to achieve
> lower latency and higher bandwidth, but I guess the investment would be low.
>
> 2. It will increase the probability of a block-chain split
>
> The convergence of the network relies on the diminishing probability of
> two honest miners creating simultaneous competing blocks chains. To
> increase the competition chain, competing blocks must be generated in
> almost simultaneously (in the same time window approximately bounded by
> the network average block propagation delay). The probability of a block
> competition decreases exponentially with the number of blocks. In fact,
> the probability of a sustained competition on ten 1-minute blocks is one
> million times lower than the probability of a competition of one
> 10-minute block. So even if the competition probability of six 1-minute
> blocks is higher than of six ten-minute blocks, this does not imply
> reducing the block rate increases this chance, but on the contrary,
> reduces it.
>
> 3, It will reduce the security of the network
>
> The security of the network is based on two facts:
> A- The miners are incentivized to extend the best chain
> B- The probability of a reversal based on a long block competition
> decreases as more confirmation blocks are appended.
> C- Renting or buying hardware to perform a 51% attack is costly.
>
> A still holds. B holds for the same amount of confirmation blocks, so 6
> confirmation blocks in a 10-minute block-chain is approximately
> equivalent to 6 confirmation blocks in a 1-minute block-chain.
> Only C changes, as renting the hashing power for 6 minutes is ten times
> less expensive as renting it for 1 hour. However, there is no shop where
> one can find 51% of the hashing power to rent right now, nor probably
> will ever be if Bitcoin succeeds. Last, you can still have a 1 hour
> confirmation (60 1-minute blocks) if you wish for high-valued payments,
> so the security decreases only if participant wish to decrease it.
>
> 4. Reducing the block propagation time on the average case is good, but
> what happen in the worse case?
>
> Most methods proposed to reduce the block propagation delay do it only
> on the average case. Any kind of block compression relies on both
> parties sharing some previous information. In the worse case it's true
> that a miner can create and try to broadcast a block that takes too much
> time to verify or bandwidth to transmit. This is currently true on the
> Bitcoin network. Nevertheless there is no such incentive for miners,
> since they will be shooting on their own foots. Peter Todd has argued
> that the best strategy for miners is actually to reach 51% of the
> network, but not more. In other words, to exclude the slowest 49%
> percent. But this strategy of creating bloated blocks is too risky in
> practice, and surely doomed to fail, as network conditions dynamically
> change. Also it would be perceived as an attack to the network, and the
> miner (if it is a public mining pool) would be probably blacklisted.
>
> 5. Thousands of SPV wallets running in mobile devices would need to be
> upgraded (thanks Mike).
>
> That depends on the current upgrade rate for SPV wallets like Bitcoin
> Wallet and BreadWallet. Suppose that the upgrade rate is 80%/year: we
> develop the source code for the change now and apply the change in Q2
> 2016, then most of the nodes will already be upgraded by when the
> hardfork takes place. Also a public notice telling people to upgrade in
> web pages, bitcointalk, SPV wallets warnings, coindesk, one year in
> advance will give plenty of time to SPV wallet users to upgrade.
>
> 6. If there are 10x more blocks, then there are 10x more block headers,
> and that increases the amount of bandwidth SPV wallets need to catch up
> with the chain
>
> A standard smartphone with average cellular downstream speed downloads
> 2.6 headers per second (1600 kbits/sec) [3], so if synchronization were
> to be done only at night when the phone is connected to the power line,
> then it would take 9 minutes to synchronize with 1440 headers/day. If a
> person should accept a payment, and the smart-phone is 1 day
> out-of-synch, then it takes less time to download all the missing
> headers than to wait for a 10-minute one block confirmation. Obviously
> all smartphones with 3G have a downstream bandwidth much higher,
> averaging 1 Mbps. So the whole synchronization will be done less than a
> 1-minute block confirmation.
>
> According to CISCO mobile bandwidth connection speed increases 20% every
> year. In four years, it will have doubled, so mobile phones with lower
> than average data connection will soon be able to catchup.
> Also there is low-hanging-fruit optimizations to the protocol that have
> not been implemented: each header is 80 bytes in length. When a set of
> chained headers is transferred, the headers could be compressed,
> stripping 32 bytes of each header that is derived from the previous
> header hash digest. So a 40% compression is already possible by slightly
> modifying the wire protocol.
>
> 7. There has been insufficient testing and/or insufficient research into
> technical/economic implications or reducing the block rate
>
> This is partially true. in the GHOST paper, this has been analyzed, and
> the problem was shown to be solvable for block intervals of just a few
> seconds. There are several proof-of-work cryptocurrencies in existence
> that have lower than 1 minute block intervals and they work just fine.
> First there was Bitcoin with a 10 minute interval, then was LiteCoin
> using a 2.5 interval, then was DogeCoin with 1 minute, and then
> QuarkCoin with just 30 seconds. Every new cryptocurrency lowers it a
> little bit. Some time ago I decided to research on the block rate to
> understand how the block interval impacts the stability and capability
> of the cryptocurrency network, and I came up with the idea of the DECOR+
> protocol [4] (which requires changes in the consensus code). In my
> research I also showed how the stale rate can be easily reduced only
> with changes in the networking code, and not in the consensus code.
> These networking optimizations ( O(1) propagation using headers-first or
> IBLTs), can be added later.
>
> Mortifying Bitcoin to accommodate the change to lower the block rate
> requires at least:
>
> - Changing the 21 BTC reward per block to 2.1 BTC
> - Changing the nPowTargetTimespan constant
> - Writing code to hard-fork automatically when the majority of miners
> have upgraded.
> - Allow transaction version 3, and interpret nLockTimes of transaction
> version 2 as being multiplied by 10.
>
> All changes comprises no more than 15 lines of code. This is much less
> than the number of lines modified by Gavin's 20Mb patch.
>
> As a conclusion, I haven't yet heard a good argument against lowering
> the block rate.
>
> Best regards,
> Sergio.
>
> [0] https://medium.com/@octskyward/the-capacity-cliff-586d1bf7715e <https://medium.com/@octskyward/the-capacity-cliff-586d1bf7715e>
> [1] https://bitslog.wordpress.com/2014/02/17/5-sec-block-interval/ <https://bitslog.wordpress.com/2014/02/17/5-sec-block-interval/>
> [2] http://gavinandresen.ninja/time-to-roll-out-bigger-blocks <http://gavinandresen.ninja/time-to-roll-out-bigger-blocks>
> [3]
> http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/white_paper_c11-520862.html <http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/white_paper_c11-520862.html>
> [4] https://bitslog.wordpress.com/2014/05/02/decor/ <https://bitslog.wordpress.com/2014/05/02/decor/>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y <http://ad.doubleclick.net/ddm/clk/290420510;117567292;y>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y_______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150511/e26992a3/attachment.html>

2023-06-07 15:34:40

- reply

📅 Original date posted:2015-05-11
📝 Original message:> On 11 May 2015, at 12:10, insecurity at national.shitposting.agency wrote:
>
> On 2015-05-11 10:34, Peter Todd wrote:
>> How do you see that blacklisting actually being done?
>
> Same way ghash.io was banned from the network when used Finney attacks
> against BetCoin Dice.
>
> As Andreas Antonopoulos says, if any of the miners do anything bad, we
> just ban them from mining. Any sort of attack like this only lasts 10
> minutes as a result. Stop worrying so much.
This doesn't work because a large-scale miner can trivially make themselves look like a very large number of much smaller scale miners. Their ability to minimize variance comes from the cumulative totals they control so 10 pools of 1% of the network cumulatively have the same variance as 1 pool with 10% of the network. It's also very easy for miners to relay blocks via different addresses and the cost is minimal. The biggest cost would be in DDoS prevention and a miner that actually split their pool into lots of small fragments would actually give themselves the ability to do quite a lot of DDoS mitigation anyway. If no-one is doing this right now it's simply because they've not had the right incentives to make it worthwhile; if the incentives make it worthwhile then this is pretty trivial to do.
This is one area where anonymity on behalf of transaction validators and block makers essentially makes it pretty-much impossible to maintain any sort of sanctions against antisocial behaviour.

2023-06-07 15:33:27

- reply

📅 Original date posted:2015-05-07
📝 Original message:> On 7 May 2015, at 11:52, Jorge Timón <jtimon at jtimon.cc> wrote:
>
> On Thu, May 7, 2015 at 11:25 AM, Mike Hearn <mike at plan99.net> wrote:
>> I observed to Wladimir and Gavin in private that this timeline meant a change to the block size was unlikely to get into 0.11, leaving only 0.12, which would give everyone only a few months to upgrade in order to fork the chain by the end of the winter growth season. That seemed tight.
>
> Can you please elaborate on what terrible things will happen if we
> don't increase the block size by winter this year?
> I assume that you are expecting full blocks by then, have you used any
> statistical technique to come up with that date or is it just your
> guess?
> Because I love wild guesses and mine is that full 1 MB blocks will not
> happen until June 2017.
I've been looking at this problem for quite a while (Gavin cited some of my work a few days ago) so thought I'd chime in with a few thoughts (some of which I've not published). I believe the major problem here is that this isn't just an engineering decision; the reaction of the miners will actually determine the success or failure of any course of action. In fact any decision forced upon them may backfire if they collectively take exception to it. It's worth bearing in mind that most of the hash rate is now under the control of relatively large companies, many of whom have investors who are expecting to see returns; it probably isn't sufficient to just expect them to "do the right thing".
We're seeing plenty of full 1M byte blocks already and have been for months. Typically whenever we have one of the large inter-block gaps then these are often followed by one (and sometimes several) completely full blocks (full by the definition of whatever the miner wanted to use as a size limit).
The problem with this particular discussion is that there are quite a few "knowns" but an equally large number of "unknowns". Let's look at them:
Known: There has been a steady trend towards the mean block size getting larger. See https://blockchain.info/charts/avg-block-size?timespan=all&showDataPoints=false&daysAverageString=7&show_header=true&scale=0&address= <https://blockchain.info/charts/avg-block-size?timespan=all&showDataPoints=false&daysAverageString=7&show_header=true&scale=0&address=>
Known: Now the trend was definitely increasing quite quickly last year but for the last few months has been slowing down, however we did see pretty much a 2x increase in mean block sizes in 2014.
Known: For most of 2015 we've actually been seeing that rate slow quite dramatically, but the total numbers of transactions are still rising so we're seeing mean transaction sizes have been reducing, and that tallies with seeing more transactions per block: https://blockchain.info/charts/n-transactions-per-block?timespan=all&showDataPoints=false&daysAverageString=7&show_header=true&scale=0&address= <https://blockchain.info/charts/n-transactions-per-block?timespan=all&showDataPoints=false&daysAverageString=7&show_header=true&scale=0&address=>
Unknown: Why are seeing more smaller transactions? Are we simply seeing more efficient use of blockchain resources or have some large consumers of block space going away? How much more block space compression might be possible in, say, the next 12 months?
Known: If we reach the point where all blocks are 1M bytes then there's a major problem in terms of transaction confirmation. I published an analysis of the impact of different mean block sizes against confirmation times: http://hashingit.com/analysis/34-bitcoin-traffic-bulletin <http://hashingit.com/analysis/34-bitcoin-traffic-bulletin>. The current 35% to 45% mean block size doesn't have a huge impact on transaction confirmations (assuming equal fees for all) but once we're up at 80% then things start to get unpleasant. Instead of 50% of first confirmations taking about 7 minutes they instead take nearer to 19 minutes.
Known: There are currently a reasonably large number of zero-fee transactions getting relayed and mined. If things start to slow down then there will be a huge incentive to delay them (or drop them altogether).
Unknown: If block space starts to get more scarce then how will this affect the use of the blockchain? Do the zero-fee TXs move to some batched transfer solution via third party? Do people start to get smarter about how TXs are encoded? Do some TXs go away completely (there are a lot of long-chain transactions that may simply be "noise" creating an artificially inflated view of transaction volumes)?
Known: There's a major problem looming for miners at the next block reward halving. Many are already in a bad place and without meaningful fees then sans a 2x increase in the USD:BTC ratio then many will simply have to leave the network, increasing centralisation risks. There seems to be a fairly pervasive assumption that the 300-ish MW of power that they currently use is going to pay for itself (ignoring capital and other operating costs).
Unknown: If the block size is increased and yet more negligible fee transactions are dumped onto the network then that might well motivate some large fraction of miners to start to clamp block sizes or reject transactions below a certain fee threshold; they can easily create their own artificial scarcity if enough of them feel it is in their interest (it's not the most tricky setting to change). One can well imagine VC investors in mining groups asking why they're essentially subsidising all of the other VC-funded Bitcoin startups.
Known: the orphan rate is still pretty-high even with everyone's fast connections. If we assume that 20M byte blocks become possible then that's likely to increase.
Unknown: What are the security implications for larger blocks (this one (at least) can be simulated though)? For example, could large blocks with huge numbers of trivial transactions be used to put other validators at a disadvantage in a variant of a selfish mining attack? I've seen objections that such bad actors could be blacklisted in the future but it's not clear to me how. A private mining pool can trivially be made to appear like 100 pools of 1% of the size without significantly affecting the economics of running that private mine.
Cheers,
Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150507/360dfd3b/attachment.html>