📅 Original date posted:2011-06-23
🗒️ Summary of this message: Instawallet is experiencing slow performance due to bitcoind taking too long to compute balances. The solution is to implement a cache system.
📝 Original message:Hi there!
Instawallet has enjoyed steady growth and I'm running into a bottleneck
now with "getbalance <someaccounthere>" taking quite some time to
complete. My understanding is, that this is because bitcoind runs
through all relevant transactions each time anew to compute the balance.
I was hoping the list could give me some pointers/ideas on how I can
improve this.
I might start to do account handling completely in my application at
some point, but for now I would like to continue letting bitcoind handle
this, so I don't have to worry about thinks like block chain reorgs.
Basically I don't have enough familiarity with the source code to feel
confident about doing fairly invasive changes. So right now I'm planning
to implement a very simple cache (directly in bitcoind), which just
caches calls to getbalance and simply invalidates the whole cache as
soon as a new block or a transaction that affects the wallet comes in.
I'm hoping this will help a little bit, although with blocks arriving
every 10 minutes on average it's not really the perfect solution.
Maybe one step better would be to only invalidate cache entries that are
affected by a new transaction or by transactions arriving in a block.
This would need to take block chain reorgs into account though, which
seems fairly complicated. Maybe I could simply invalidate the whole
cache on reorgs, which should be seldom enough that it's not a big
problem. Where would be a good place in the source code to hook into to
notice block chain reorgs?
The perfect solution would be, of course, if bitcoind could keep running
balances of all accounts and update them as new information becomes
available. But as I said, I don't feel that I have a good overview of
all the corner cases to make such an improvement. I guess an extensive
test suite testing all sorts of esoteric block chain reorgs would really
be a nice thing to have. :-)
Regards!
Jan
jan at uos.de [ARCHIVE] /
npub1q4…x8aml
2023-06-07 01:22:38
in reply to nevent1q…ptlp
Author Public Key
npub1q4mxmcaamghs49c995y7xj5a9ndnrnnqm4z8rjaj0t9amc9uw4dsmx8amlPublished at
2023-06-07 01:22:38Event JSON
{
"id": "cd0fc5d80a3cce7fd2f0f8e3baafab786c1bd3f7ff2b767d136c5c81a463116e",
"pubkey": "05766de3bdda2f0a97052d09e34a9d2cdb31ce60dd4471cbb27acbdde0bc755b",
"created_at": 1686100958,
"kind": 1,
"tags": [
[
"e",
"0b3fc1a7bb6f39c49e0e9d490e6ea1c87daab50dea07f3a12027be454b102ce6",
"",
"reply"
],
[
"p",
"857b83d2624bdaeb3fa001fda56aa163e974359f543a471b9881d6660ad2bf98"
]
],
"content": "📅 Original date posted:2011-06-23\n🗒️ Summary of this message: Instawallet is experiencing slow performance due to bitcoind taking too long to compute balances. The solution is to implement a cache system.\n📝 Original message:Hi there!\n\nInstawallet has enjoyed steady growth and I'm running into a bottleneck\nnow with \"getbalance \u003csomeaccounthere\u003e\" taking quite some time to\ncomplete. My understanding is, that this is because bitcoind runs\nthrough all relevant transactions each time anew to compute the balance.\nI was hoping the list could give me some pointers/ideas on how I can\nimprove this.\n\nI might start to do account handling completely in my application at\nsome point, but for now I would like to continue letting bitcoind handle\nthis, so I don't have to worry about thinks like block chain reorgs.\n\nBasically I don't have enough familiarity with the source code to feel\nconfident about doing fairly invasive changes. So right now I'm planning\nto implement a very simple cache (directly in bitcoind), which just\ncaches calls to getbalance and simply invalidates the whole cache as\nsoon as a new block or a transaction that affects the wallet comes in.\nI'm hoping this will help a little bit, although with blocks arriving\nevery 10 minutes on average it's not really the perfect solution.\n\nMaybe one step better would be to only invalidate cache entries that are\naffected by a new transaction or by transactions arriving in a block.\nThis would need to take block chain reorgs into account though, which\nseems fairly complicated. Maybe I could simply invalidate the whole\ncache on reorgs, which should be seldom enough that it's not a big\nproblem. Where would be a good place in the source code to hook into to\nnotice block chain reorgs?\n\nThe perfect solution would be, of course, if bitcoind could keep running\nbalances of all accounts and update them as new information becomes\navailable. But as I said, I don't feel that I have a good overview of\nall the corner cases to make such an improvement. I guess an extensive\ntest suite testing all sorts of esoteric block chain reorgs would really\nbe a nice thing to have. :-)\n\nRegards!\nJan",
"sig": "44e463a0be1cc6334bc7c151e626424695590c2716a6c5146aa65c339be829059b860c57117b5b50adf4e46181e673af76b0242ce1171862382cf0a381d52eb0"
}