📅 Original date posted:2017-12-26
📝 Original message:I have needed to re-tac my intentions somewhat, there is still much
work to be done.
This is a request for assistance and further discussion of the re-
revised proposal. I am sure there are still issues to be resolved.
## BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering
Transactions In Blocks
Schema:
##########
Document: BIP Proposal
Title: UTPFOTIB - Use Transaction Priority For Ordering Transactions In
Blocks
Date: 26-12-2017
Author: Damian Williamson <willtech at live.com.au>
Licence: Creative Commons Attribution-ShareAlike 4.0 International
License.
URL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-
utpfotib-use-transaction-priority-for-order
##########
### 1. Abstract
This document proposes to address the issue of transactional
reliability in Bitcoin, where valid transactions may be stuck in the
transaction pool for extended periods or never confirm.
There are two key issues to be resolved to achieve this:
1. The current transaction bandwidth limit.
2. The current ad-hoc methods of including transactions in blocks
resulting in variable and confusing confirmation times for valid
transactions, including transactions with a valid fee that may never
confirm.
It is important with any change to protect the value of fees as these
will eventually be the only payment that miners receive. Rather than an
auction model for limited bandwidth, the proposal results in a fee for
priority service auction model.
It would not be true to suggest that all feedback received so far has
been entirely positive although, most of it has been constructive.
The previous threads for this proposal are available here:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/s
ubject.html
In all parts of this proposal, references to a transaction, a valid
transaction, a transaction with a valid fee, a valid fee, etc. is
defined as any transaction that is otherwise valid with a fee of at
least 0.00001000 BTC/KB as defined as the dust level, interpreting from
Bitcoin Core GUI. Transactions with a fee lower than this rate are
considered dust.
In all parts of this proposal, dust and zero-fee transactions are
always ignored and/or excluded unless specifically mentioned.
It is generally assumed that miners currently prefer to include
transactions with higher fees.
### 2. The need for this proposal
We all must learn to admit that transaction bandwidth is still lurking
as a serious issue for the operation, reliability, safety, consumer
acceptance, uptake and, for the value of Bitcoin.
I recently sent a payment which was not urgent so; I chose three-day
target confirmation from the fee recommendation. That transaction has
still not confirmed after now more than six days - even waiting twice
as long seems quite reasonable to me (note for accuracy: it did
eventually confirm). That transaction is a valid transaction; it is not
rubbish, junk or, spam. Under the current model with transaction
bandwidth limitation, the longer a transaction waits, the less likely
it is ever to confirm due to rising transaction numbers and being
pushed back by transactions with rising fees.
I argue that no transactions with fees above the dust level are rubbish
or junk, only some zero fee transactions might be spam. Having an ever-
increasing number of valid transactions that do not confirm as more new
transactions with higher fees are created is the opposite of operating
a robust, reliable transaction system.
Business cannot operate with a model where transactions may or may not
confirm. Even a business choosing a modest fee has no guarantee that
their valid transaction will not be shuffled down by new transactions
to the realm of never confirming after it is created. Consumers also
will not accept this model as Bitcoin expands. If Bitcoin cannot be a
reliable payment system for confirmed transactions then consumers, by
and large, will simply not accept the model once they understand.
Bitcoin will be a dirty payment system, and this will kill the value of
Bitcoin.
Under the current system, a minority of transactions will eventually be
the lucky few who have fees high enough to escape being pushed down the
list.
Once there are more than x transactions (transaction bandwidth limit)
every ten minutes, only those choosing twenty-minute confirmation (2
blocks) from the fee recommendations will have initially at most a
fifty percent chance of ever having their payment confirm when 2x
transactions is reached. Presently, not even using fee recommendations
can ensure a sufficiently high fee is paid to ensure transaction
confirmation.
I also argue that the current auction model for limited transaction
bandwidth is wrong, is not suitable for a reliable transaction system
and, is wrong for Bitcoin. All transactions with valid fees must
confirm in due time. Currently, Bitcoin is not a safe way to send
payments.
I do not believe that consumers and business are against paying fees,
even high fees. What is required is operational reliability.
This great issue needs to be resolved for the safety and reliability of
Bitcoin. The time to resolve issues in commerce is before they become
great big issues. The time to resolve this issue is now. We must have
the foresight to identify and resolve problems before they trip us
over. Simply doubling block sizes every so often is reactionary and is
not a reliable permanent solution.
I have written this proposal for a technical solution but, need your
help to write it up to an acceptable standard to be a full BIP.
### 3. The problem
Everybody wants value. Miners want to maximise revenue from fees (and
we presume, to minimise block size). Consumers need transaction
reliability and, (we presume) want low fees.
The current transaction bandwidth limit is a limiting factor for both.
As the operational safety of transactions is limited, so is consumer
confidence as they realise the issue and, accordingly, uptake is
limited. Fees are artificially inflated due to bandwidth limitations
while failing to provide a full confirmation service for all valid
transactions.
Current fee recommendations provide no satisfaction for transaction
reliability and, as Bitcoin scales, this will worsen.
Transactions are included in blocks by miners using whatever basis they
prefer. We expect that this is usually a fee-based priority. However,
even transactions with a valid fee may be left in the transaction pool
for some time. As transaction bandwidth becomes an issue, not even
extreme fees can ensure a transaction is processed in a timely manner
or at all.
Bitcoin must be a fully scalable and reliable service, providing full
transaction confirmation for every valid transaction.
The possibility to send a transaction with a fee lower than one that is
acceptable to allow eventual transaction confirmation should be removed
from the protocol and also from the user interface.
### 4. Solution summary
#### Main solution
Provide each valid transaction in the mempool with an individual
transaction priority each time before choosing transactions to include
in the current block. The priority being a function of the fee (on a
curve), and the time waiting in the transaction pool (also on a curve)
out to n days (n = 60 days ?), and extending past n days. The value for
fee on a curve may need an upper limit. The transaction priority to
serve as the likelihood of a transaction being included in the current
block, and for determining the order in which transactions are tried to
see if they will be included.
Nodes will need to keep track of when a transaction is first seen. It
is satisfactory for each node to do this independently provided the
information survives node restart. If there is a more reliable way to
determine when a transaction was first seen on the network then it
should be utilised.
Use a dynamic target block size to make the current block. If the block
size is consistently too small then I expect ageing transactions will
be overrepresented as a portion of the block contents, to the point
where blocks will only contain the oldest transactions as they age past
n days. If block size is too large on average then this will shrink the
transaction pool. Determine the target block size using; pre-
rollout(current average valid transaction pool size) x ( 1 / (144 x n
days ) ) = number of transactions to be included in the current block.
The block created should be a minimum 1MB in size regardless if the
target block size is lower.
Nodes that have not yet adopted the proposal will just continue to
create 1MB unordered blocks.
The default value for mempoolexpiry may in future need to be adjusted
to match n days or, perhaps using less than n = 14 days may be a more
sensible approach?
All block created with dynamic size should be verified to ensure
conformity to a probability distribution curve resulting from the
priority method. Since the input is a probability, the output should
conform to a probability distribution.
The curves used for the priority of transactions would have to be
appropriate. Perhaps a mathematician with experience in probability can
develop the right formulae. My thinking is a steep curve. I suppose
that the probability of all transactions should probably account for a
sufficient number of inclusions that the target block size is met on
average although, it may not always be. As a suggestion, consider
including some dust or zero-fee transactions to pad if each valid
transaction is tried and the target block size is not yet met, highest
BTC transaction value first?
**Explanation of the operation of priority:**
> If transaction priority is, for example, a number between one (low)
and one-hundred (high) it can be directly understood as the percentage
chance in one-hundred of a transaction being included in the block.
Using probability or likelihood infers that there is some function of
random. Try the transactions in priority order from highest to lowest,
if random (100) < transaction priority then the transaction is included
until the target block size is met.
> To break it down further, if both the fee on a curve value and the
time waiting on a curve value are each a number between one and one-
hundred, a rudimentary method may be to simply multiply those two
numbers, to find the priority number. For example, a middle fee
transaction waiting thirty days (if n = 60 days) may have a value of
five for each part (yes, just five, the values are on a curve). When
multiplied that will give a priority value of twenty-five, or, a
twenty-five percent chance at that moment of being included in the
block; it will likely be included in one of the next four blocks,
getting more likely each chance. If it is still not included then the
value of time waiting will be higher, making for more probability. A
very low fee transaction would have a value for the fee of one. It
would not be until near sixty-days that the particular low fee
transaction has a high likelihood of being included in the block.
In practice it may be more useful to use numbers representative of one-
hundred for the highest fee priority curve down to a small fraction of
one for the lowest fee and, from one for a newly seen transaction up to
a proportionately high number above one-hundred for the time waiting
curve. It is truely beyond my level of math to resolve probability
curves accurately without much trial and error.
The primary reason for addressing the issue is to ensure transactional
reliability and scalability while having each valid transaction confirm
in due time.
#### Pros
* Maximizes transaction reliability.
* Overcomes transaction bandwidth limit.
* Fully scalable.
* Maximizes possibility for consumer and business uptake.
* Maximizes total fees paid per block without reducing reliability;
because of reliability, in time confidence and overall uptake are
greater; therefore, more transactions.
* Market determines fee paid for transaction priority.
* Fee recommendations work all the way out to 30 days or greater.
* Provides additional block entropy; greater security since there is
less probability of predicting the next block. _Although this is not
necessary it is a product of the operation of this proposal._
#### Cons
* Could initially lower total transaction fees per block.
* Must be first be programmed.
#### Pre-rollout
Nodes need to have at a minimum a loose understanding of the average
(since there is no consensus) size of the transaction pool as a
requirement to enable future changes to the way blocks are constructed.
A new network service should be constructed to meet this need. This
service makes no changes to any existing operation or function of the
node. Initially, Bitcoin Core is a suitable candidate.
**The service must:**
* Have an individual temporary (runtime permanent only) Serial Node
ID.
* Accept communication of the number of valid transactions in the
mempool of another valid Bitcoin node along with the Serial Node ID of
the node whose value is provided.
* Disconnect the service from any non-Bitcoin node. Bitcoin Core may
handle this already?
* Expire any value not updated for k minutes (k = 30 minutes?).
* Broadcast all mempool information the node has every m minutes (m =
10 minutes?), including its own.
* Nodes own mempool information should not be broadcast or used in
calculation until the node has been up long enough for the mempool to
normalise for at least o minutes (o = 300 minutes ?)
* Only new or updated mempool values should be transmitted to the
same node. Updated includes updated with no change.
* All known mempool information must survive node restart.
* If the nodes own mempool is not normalised and network information
is not available to calculate an average just display zero.
* Internally, the average transaction pool size must return the
calculated average if an average is available or, if none is available
just the number of valid transactions in the node's own mempool
regardless if it is normalised.
Bitcoin Core must use all collated information on mempool size to
calculate a figure for the average mempool size.
The calculated figure should be displayed in the appropriate place in
the Debug window alongside the text Network average transactions.
Consideration must be given before development of the network bandwidth
this would require. All programming must be consistent with the current
operation and conventions of Bitcoin Core. Methods must work on all
platforms.
As this new service does not affect any existing service or feature of
Bitcoin or Bitcoin Core, this can technically be programmed now and
included in Bitcoin Core at any time.
### 5. Solution operation
This is a simplistic view of the operation. The actual operation will
need to be determined accurately in a spec for the programmer.
1. Determine the target block size for the current block.
2. Assign a transaction priority to each valid transaction in the
mempool.
3. Select transactions to include in the current block using
probability in transaction priority order until the target block size
is met. If target block size is not met, include dust and zero-fee
transactions to pad.
4. Solve block.
5. Broadcast the current block when it is solved.
6. Block is received.
7. Block verification process.
8. Accept/reject block based on verification result.
9. Repeat.
### 6. Closing comments
It may be possible to verify blocks conform to the proposal by showing
that the probability for all transactions included in the block
statistically conforms to a probability distribution curve, *if* the
individual transaction priority can be recreated. I am not that deep
into the mathematics; however, it may also be possible to use a similar
method to do this just based on the fee, that statistically, the block
conforms to a fee distribution. Any dust and zero-fee transactions
would have to be ignored. This solution needs a competent mathematician
with experience in probability and statistical distribution.
There has been some concern expressed over spam and very low fee
transactions, and an infinite block size resulting. I hope that for
those concerned using the dust level addresses the issue, especially as
the value of Bitcoin grows.
This proposal is necessary. I implore, at the very least, that we use
some method that validates full transaction reliability and enables
scalability of Bitcoin. If not this proposal, an alternative.
I have done as much with this proposal as I feel that I am able so far
but continue to take your feedback.
Regards,
Damian Williamson
[](http://creativecommons.org/licenses/by-sa/4.0/)
<span xmlns:dct="http://purl.org/dc/terms/";
href="http://purl.org/dc/dcmitype/Text"; property="dct:title"
rel="dct:type">BIP Proposal: UTPFOTIB - Use Transaction Priority For
Ordering Transactions In Blocks</span> by [Damian Williamson
<willtech at live.com.au>](http://thekingjameshrmh.tumblr.com/post/1
68948530950/bip-proposal-utpfotib-use-transaction-priority-for-order)
is licensed under a [Creative Commons Attribution-ShareAlike 4.0
International License](http://creativecommons.org/licenses/by-sa/4.0/).
Based on a work at [https://lists.linuxfoundation.org/pipermail/bitcoin
-dev/2017-
December/015371.html](https://lists.linuxfoundation.org/pipermail/bitco
in-dev/2017-December/015371.html).
Permissions beyond the scope of this license may be available at [https
://opensource.org/licenses/BSD-3-
Clause](https://opensource.org/licenses/BSD-3-Clause).
Damian Williamson [ARCHIVE] /
npub1sp…8lz2h
2023-06-07 18:08:31
in reply to nevent1q…a2nc
Author Public Key
npub1spe4rkz350zpn7h2pz07t8sdwgt440zmx9k530afwj6gmq7q0lksh8lz2hPublished at
2023-06-07 18:08:31Event JSON
{
"id": "7a9e7c7fedfef896715e25aa97a42395a8a3db8599ef3855b4fec89950029e2a",
"pubkey": "807351d851a3c419faea089fe59e0d72175abc5b316d48bfa974b48d83c07fed",
"created_at": 1686161311,
"kind": 1,
"tags": [
[
"e",
"f793970afb109cf4a5a76a12730a5f14fd9b13d6742084777f8dcf62b6b2bfea",
"",
"root"
],
[
"e",
"c36a43e0024383a1e68f0f8783e8591499977844b656b7a46b37e13770e5dd81",
"",
"reply"
],
[
"p",
"7b56ce909050fc091d961647a9fc1b551f9f5e7b15a9b6d9ae68f69eb37e5102"
]
],
"content": "📅 Original date posted:2017-12-26\n📝 Original message:I have needed to re-tac my intentions somewhat, there is still much\nwork to be done.\n\nThis is a request for assistance and further discussion of the re-\nrevised proposal. I am sure there are still issues to be resolved.\n\n## BIP Proposal: UTPFOTIB - Use Transaction Priority For Ordering\nTransactions In Blocks\n\nSchema: \n########## \nDocument: BIP Proposal \nTitle: UTPFOTIB - Use Transaction Priority For Ordering Transactions In\nBlocks \nDate: 26-12-2017 \nAuthor: Damian Williamson \u0026lt;willtech at live.com.au\u0026gt; \nLicence: Creative Commons Attribution-ShareAlike 4.0 International\nLicense. \nURL: http://thekingjameshrmh.tumblr.com/post/168948530950/bip-proposal-\nutpfotib-use-transaction-priority-for-order \n########## \n\n### 1. Abstract\n\nThis document proposes to address the issue of transactional\nreliability in Bitcoin, where valid transactions may be stuck in the\ntransaction pool for extended periods or never confirm.\n\nThere are two key issues to be resolved to achieve this:\n\n1. The current transaction bandwidth limit.\n2. The current ad-hoc methods of including transactions in blocks\nresulting in variable and confusing confirmation times for valid\ntransactions, including transactions with a valid fee that may never\nconfirm.\n\nIt is important with any change to protect the value of fees as these\nwill eventually be the only payment that miners receive. Rather than an\nauction model for limited bandwidth, the proposal results in a fee for\npriority service auction model.\n\nIt would not be true to suggest that all feedback received so far has\nbeen entirely positive although, most of it has been constructive.\n\nThe previous threads for this proposal are available here: \nhttps://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/s\nubject.html\n\nIn all parts of this proposal, references to a transaction, a valid\ntransaction, a transaction with a valid fee, a valid fee, etc. is\ndefined as any transaction that is otherwise valid with a fee of at\nleast 0.00001000 BTC/KB as defined as the dust level, interpreting from\nBitcoin Core GUI. Transactions with a fee lower than this rate are\nconsidered dust.\n\nIn all parts of this proposal, dust and zero-fee transactions are\nalways ignored and/or excluded unless specifically mentioned.\n\nIt is generally assumed that miners currently prefer to include\ntransactions with higher fees.\n\n### 2. The need for this proposal\n\nWe all must learn to admit that transaction bandwidth is still lurking\nas a serious issue for the operation, reliability, safety, consumer\nacceptance, uptake and, for the value of Bitcoin.\n\nI recently sent a payment which was not urgent so; I chose three-day\ntarget confirmation from the fee recommendation. That transaction has\nstill not confirmed after now more than six days - even waiting twice\nas long seems quite reasonable to me (note for accuracy: it did\neventually confirm). That transaction is a valid transaction; it is not\nrubbish, junk or, spam. Under the current model with transaction\nbandwidth limitation, the longer a transaction waits, the less likely\nit is ever to confirm due to rising transaction numbers and being\npushed back by transactions with rising fees.\n\nI argue that no transactions with fees above the dust level are rubbish\nor junk, only some zero fee transactions might be spam. Having an ever-\nincreasing number of valid transactions that do not confirm as more new\ntransactions with higher fees are created is the opposite of operating\na robust, reliable transaction system.\n\nBusiness cannot operate with a model where transactions may or may not\nconfirm. Even a business choosing a modest fee has no guarantee that\ntheir valid transaction will not be shuffled down by new transactions\nto the realm of never confirming after it is created. Consumers also\nwill not accept this model as Bitcoin expands. If Bitcoin cannot be a\nreliable payment system for confirmed transactions then consumers, by\nand large, will simply not accept the model once they understand.\nBitcoin will be a dirty payment system, and this will kill the value of\nBitcoin.\n\nUnder the current system, a minority of transactions will eventually be\nthe lucky few who have fees high enough to escape being pushed down the\nlist.\n\nOnce there are more than x transactions (transaction bandwidth limit)\nevery ten minutes, only those choosing twenty-minute confirmation (2\nblocks) from the fee recommendations will have initially at most a\nfifty percent chance of ever having their payment confirm when 2x\ntransactions is reached. Presently, not even using fee recommendations\ncan ensure a sufficiently high fee is paid to ensure transaction\nconfirmation.\n\nI also argue that the current auction model for limited transaction\nbandwidth is wrong, is not suitable for a reliable transaction system\nand, is wrong for Bitcoin. All transactions with valid fees must\nconfirm in due time. Currently, Bitcoin is not a safe way to send\npayments.\n\nI do not believe that consumers and business are against paying fees,\neven high fees. What is required is operational reliability.\n\nThis great issue needs to be resolved for the safety and reliability of\nBitcoin. The time to resolve issues in commerce is before they become\ngreat big issues. The time to resolve this issue is now. We must have\nthe foresight to identify and resolve problems before they trip us\nover. Simply doubling block sizes every so often is reactionary and is\nnot a reliable permanent solution.\n\nI have written this proposal for a technical solution but, need your\nhelp to write it up to an acceptable standard to be a full BIP.\n\n### 3. The problem\n\nEverybody wants value. Miners want to maximise revenue from fees (and\nwe presume, to minimise block size). Consumers need transaction\nreliability and, (we presume) want low fees.\n\nThe current transaction bandwidth limit is a limiting factor for both.\nAs the operational safety of transactions is limited, so is consumer\nconfidence as they realise the issue and, accordingly, uptake is\nlimited. Fees are artificially inflated due to bandwidth limitations\nwhile failing to provide a full confirmation service for all valid\ntransactions.\n\nCurrent fee recommendations provide no satisfaction for transaction\nreliability and, as Bitcoin scales, this will worsen.\n\nTransactions are included in blocks by miners using whatever basis they\nprefer. We expect that this is usually a fee-based priority. However,\neven transactions with a valid fee may be left in the transaction pool\nfor some time. As transaction bandwidth becomes an issue, not even\nextreme fees can ensure a transaction is processed in a timely manner\nor at all.\n\nBitcoin must be a fully scalable and reliable service, providing full\ntransaction confirmation for every valid transaction.\n\nThe possibility to send a transaction with a fee lower than one that is\nacceptable to allow eventual transaction confirmation should be removed\nfrom the protocol and also from the user interface.\n\n### 4. Solution summary\n\n#### Main solution\n\nProvide each valid transaction in the mempool with an individual\ntransaction priority each time before choosing transactions to include\nin the current block. The priority being a function of the fee (on a\ncurve), and the time waiting in the transaction pool (also on a curve)\nout to n days (n = 60 days ?), and extending past n days. The value for\nfee on a curve may need an upper limit. The transaction priority to\nserve as the likelihood of a transaction being included in the current\nblock, and for determining the order in which transactions are tried to\nsee if they will be included.\n\nNodes will need to keep track of when a transaction is first seen. It\nis satisfactory for each node to do this independently provided the\ninformation survives node restart. If there is a more reliable way to\ndetermine when a transaction was first seen on the network then it\nshould be utilised.\n\nUse a dynamic target block size to make the current block. If the block\nsize is consistently too small then I expect ageing transactions will\nbe overrepresented as a portion of the block contents, to the point\nwhere blocks will only contain the oldest transactions as they age past\nn days. If block size is too large on average then this will shrink the\ntransaction pool. Determine the target block size using; pre-\nrollout(current average valid transaction pool size) x ( 1 / (144 x n\ndays ) ) = number of transactions to be included in the current block.\nThe block created should be a minimum 1MB in size regardless if the\ntarget block size is lower.\n\nNodes that have not yet adopted the proposal will just continue to\ncreate 1MB unordered blocks.\n\nThe default value for mempoolexpiry may in future need to be adjusted\nto match n days or, perhaps using less than n = 14 days may be a more\nsensible approach?\n\nAll block created with dynamic size should be verified to ensure\nconformity to a probability distribution curve resulting from the\npriority method. Since the input is a probability, the output should\nconform to a probability distribution.\n\nThe curves used for the priority of transactions would have to be\nappropriate. Perhaps a mathematician with experience in probability can\ndevelop the right formulae. My thinking is a steep curve. I suppose\nthat the probability of all transactions should probably account for a\nsufficient number of inclusions that the target block size is met on\naverage although, it may not always be. As a suggestion, consider\nincluding some dust or zero-fee transactions to pad if each valid\ntransaction is tried and the target block size is not yet met, highest\nBTC transaction value first?\n\n**Explanation of the operation of priority:**\n\n\u003e If transaction priority is, for example, a number between one (low)\nand one-hundred (high) it can be directly understood as the percentage\nchance in one-hundred of a transaction being included in the block.\nUsing probability or likelihood infers that there is some function of\nrandom. Try the transactions in priority order from highest to lowest,\nif random (100) \u003c transaction priority then the transaction is included\nuntil the target block size is met.\n\n\u003e To break it down further, if both the fee on a curve value and the\ntime waiting on a curve value are each a number between one and one-\nhundred, a rudimentary method may be to simply multiply those two\nnumbers, to find the priority number. For example, a middle fee\ntransaction waiting thirty days (if n = 60 days) may have a value of\nfive for each part (yes, just five, the values are on a curve). When\nmultiplied that will give a priority value of twenty-five, or, a\ntwenty-five percent chance at that moment of being included in the\nblock; it will likely be included in one of the next four blocks,\ngetting more likely each chance. If it is still not included then the\nvalue of time waiting will be higher, making for more probability. A\nvery low fee transaction would have a value for the fee of one. It\nwould not be until near sixty-days that the particular low fee\ntransaction has a high likelihood of being included in the block.\n\nIn practice it may be more useful to use numbers representative of one-\nhundred for the highest fee priority curve down to a small fraction of\none for the lowest fee and, from one for a newly seen transaction up to\na proportionately high number above one-hundred for the time waiting\ncurve. It is truely beyond my level of math to resolve probability\ncurves accurately without much trial and error.\n\nThe primary reason for addressing the issue is to ensure transactional\nreliability and scalability while having each valid transaction confirm\nin due time.\n\n#### Pros\n\n* Maximizes transaction reliability.\n* Overcomes transaction bandwidth limit.\n* Fully scalable.\n* Maximizes possibility for consumer and business uptake.\n* Maximizes total fees paid per block without reducing reliability;\nbecause of reliability, in time confidence and overall uptake are\ngreater; therefore, more transactions.\n* Market determines fee paid for transaction priority.\n* Fee recommendations work all the way out to 30 days or greater.\n* Provides additional block entropy; greater security since there is\nless probability of predicting the next block. _Although this is not\nnecessary it is a product of the operation of this proposal._\n\n#### Cons\n\n* Could initially lower total transaction fees per block.\n* Must be first be programmed.\n\n#### Pre-rollout\n\nNodes need to have at a minimum a loose understanding of the average\n(since there is no consensus) size of the transaction pool as a\nrequirement to enable future changes to the way blocks are constructed.\n\nA new network service should be constructed to meet this need. This\nservice makes no changes to any existing operation or function of the\nnode. Initially, Bitcoin Core is a suitable candidate.\n\n**The service must:**\n\n* Have an individual temporary (runtime permanent only) Serial Node\nID.\n* Accept communication of the number of valid transactions in the\nmempool of another valid Bitcoin node along with the Serial Node ID of\nthe node whose value is provided.\n* Disconnect the service from any non-Bitcoin node. Bitcoin Core may\nhandle this already?\n* Expire any value not updated for k minutes (k = 30 minutes?).\n* Broadcast all mempool information the node has every m minutes (m =\n10 minutes?), including its own.\n* Nodes own mempool information should not be broadcast or used in\ncalculation until the node has been up long enough for the mempool to\nnormalise for at least o minutes (o = 300 minutes ?)\n* Only new or updated mempool values should be transmitted to the\nsame node. Updated includes updated with no change.\n* All known mempool information must survive node restart.\n* If the nodes own mempool is not normalised and network information\nis not available to calculate an average just display zero.\n* Internally, the average transaction pool size must return the\ncalculated average if an average is available or, if none is available\njust the number of valid transactions in the node's own mempool\nregardless if it is normalised.\n\nBitcoin Core must use all collated information on mempool size to\ncalculate a figure for the average mempool size.\n\nThe calculated figure should be displayed in the appropriate place in\nthe Debug window alongside the text Network average transactions.\n\nConsideration must be given before development of the network bandwidth\nthis would require. All programming must be consistent with the current\noperation and conventions of Bitcoin Core. Methods must work on all\nplatforms.\n\nAs this new service does not affect any existing service or feature of\nBitcoin or Bitcoin Core, this can technically be programmed now and\nincluded in Bitcoin Core at any time.\n\n### 5. Solution operation\n\nThis is a simplistic view of the operation. The actual operation will\nneed to be determined accurately in a spec for the programmer.\n\n1. Determine the target block size for the current block.\n2. Assign a transaction priority to each valid transaction in the\nmempool.\n3. Select transactions to include in the current block using\nprobability in transaction priority order until the target block size\nis met. If target block size is not met, include dust and zero-fee\ntransactions to pad.\n4. Solve block.\n5. Broadcast the current block when it is solved.\n6. Block is received.\n7. Block verification process.\n8. Accept/reject block based on verification result.\n9. Repeat.\n\n### 6. Closing comments\n\nIt may be possible to verify blocks conform to the proposal by showing\nthat the probability for all transactions included in the block\nstatistically conforms to a probability distribution curve, *if* the\nindividual transaction priority can be recreated. I am not that deep\ninto the mathematics; however, it may also be possible to use a similar\nmethod to do this just based on the fee, that statistically, the block\nconforms to a fee distribution. Any dust and zero-fee transactions\nwould have to be ignored. This solution needs a competent mathematician\nwith experience in probability and statistical distribution.\n\nThere has been some concern expressed over spam and very low fee\ntransactions, and an infinite block size resulting. I hope that for\nthose concerned using the dust level addresses the issue, especially as\nthe value of Bitcoin grows.\n\nThis proposal is necessary. I implore, at the very least, that we use\nsome method that validates full transaction reliability and enables\nscalability of Bitcoin. If not this proposal, an alternative.\n\nI have done as much with this proposal as I feel that I am able so far\nbut continue to take your feedback.\n\nRegards, \nDamian Williamson\n\n[](http://creativecommons.org/licenses/by-sa/4.0/) \n\u003cspan xmlns:dct=\"http://purl.org/dc/terms/\"\nhref=\"http://purl.org/dc/dcmitype/Text\" property=\"dct:title\"\nrel=\"dct:type\"\u003eBIP Proposal: UTPFOTIB - Use Transaction Priority For\nOrdering Transactions In Blocks\u003c/span\u003e by [Damian Williamson\n\u0026lt;willtech at live.com.au\u0026gt;](http://thekingjameshrmh.tumblr.com/post/1\n68948530950/bip-proposal-utpfotib-use-transaction-priority-for-order)\nis licensed under a [Creative Commons Attribution-ShareAlike 4.0\nInternational License](http://creativecommons.org/licenses/by-sa/4.0/).\nBased on a work at [https://lists.linuxfoundation.org/pipermail/bitcoin\n-dev/2017-\nDecember/015371.html](https://lists.linuxfoundation.org/pipermail/bitco\nin-dev/2017-December/015371.html).\nPermissions beyond the scope of this license may be available at [https\n://opensource.org/licenses/BSD-3-\nClause](https://opensource.org/licenses/BSD-3-Clause).",
"sig": "5abc5fc25a9711c7c11b48d394cefcf99ec3ecca7ef1b71aa880c0f2334f069e293c22f4e3ded9c67e7bece438b6860d4c91f4fbba662f3de95515601b0bd55b"
}