📅 Original date posted:2017-04-07
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On 04/07/2017 02:44 PM, Tomas via bitcoin-dev wrote:
> Hi Eric,
>
> On Fri, Apr 7, 2017, at 21:55, Eric Voskuil via bitcoin-dev wrote:
>> Optimization for lower memory platforms then becomes a process
>> of reducing the need for paging. This is the purpose of a cache.
>> The seam between disk and memory can be filled quite nicely by a
>> small amount of cache. On high RAM systems any cache is actually
>> a de-optimization but on low RAM systems it can prevent excessive
>> paging. This is directly analogous to a CPU cache.
>
>
> I am not entirely sure I agree with that, or understand it
> correctly.
>
> If -for example - the data of some application is a set of
> records which can be sorted from least frequently used to most
> frequently used then doing just that sort will beat any
> application-layer cache. Regardless of size of data and size of
> RAM, you simply allow the OS to use disk caching or memory map
> caching to work its magic .
It's a reasonable assumption, and given that the no-explicit-cache
implementation is a subset of the optionally-cached implementation,
was of course the initial implementation.
> In fact, I would argue that an application-layer cache *only*
> makes sense if the data model shows a *hard* distinction between
> often and not often used data. If usage-frequency is a continuous
> line, caching is best left to the OS by focussing on proper spatial
> and temporal locality of reference of your data, because the OS has
> much more information to make the right decision.
In practice this is not the case. The Bitcoin data model is neither
continuous nor strictly segregated by usage.
It is true that with sufficient RAM a cache is totally
counterproductive. It is also my experience that an independent UTXO
store is not a reasonable/necessary trade of disk space, memory
scalability, and/or code complexity in exchange for speed.
But on lower memory systems a explicit cache is beneficial. The
difference is clearly measurable in production code by simply changing
the cache limit and testing on various configurations.
e
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQEcBAEBCAAGBQJY6CXnAAoJEDzYwH8LXOFOf0YH/2qk3hYC6iEDW/DWM2ffkdb9
QM7A29Pvbfw9Wjr5Xx+ugIQvlAr4T+nByOCT6AnrqNU5K3UUmbC0KIB1rEL94hsK
QYVlLs0cOrjg8qKJpck+wcgiWw3VbEa/Y44hK7NLUxoy2HsLYaxPhqFH3GGgowqR
syga626jf2YUyudZxj1gFuqn7grkwghnzdrEUJMcqQo8IvCqjftGXlKxBGyB/AIs
Dx+5EWO9Q9IxrNpg/fsKKB6xkMxkmSx2hbD7dmEBvi/afbVF66rDTinjInG/LCju
pV7kT/GAWqGQGku6sQyAOexsxVhWA8EA/QEjvbyyGb+3YnR0s6nPk+CxO+RkOgo=
=e+Pr
-----END PGP SIGNATURE-----
Eric Voskuil [ARCHIVE] /
npub1sg…ppx3c
2023-06-07 17:59:40
in reply to nevent1q…ldjt
Author Public Key
npub1sgs97fe0n9wehe6zw7drcxdz4cy9yt9pfqjv8gasz5jlk4zezc0quppx3cPublished at
2023-06-07 17:59:40Event JSON
{
"id": "465f2261b70e2a6aef2354fa0680b58ff5cccaf64fc088b4452d48e306d5bb8b",
"pubkey": "82205f272f995d9be742779a3c19a2ae08522ca14824c3a3b01525fb5459161e",
"created_at": 1686160780,
"kind": 1,
"tags": [
[
"e",
"d4a682be1f6603f0ff8798c52b7225cac6554e21f3aedb0c80e7d41cf71983ad",
"",
"root"
],
[
"e",
"c5edef772f1e61c7cfe3c5de4e5e197bb7d9e93ef58ba90ba4997d1ccfde9805",
"",
"reply"
],
[
"p",
"1c03575343555d1132a621c49466190d680da4a306ba8b992e8b87e267609cdd"
]
],
"content": "📅 Original date posted:2017-04-07\n📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA256\n\nOn 04/07/2017 02:44 PM, Tomas via bitcoin-dev wrote:\n\u003e Hi Eric,\n\u003e \n\u003e On Fri, Apr 7, 2017, at 21:55, Eric Voskuil via bitcoin-dev wrote:\n\u003e\u003e Optimization for lower memory platforms then becomes a process\n\u003e\u003e of reducing the need for paging. This is the purpose of a cache.\n\u003e\u003e The seam between disk and memory can be filled quite nicely by a\n\u003e\u003e small amount of cache. On high RAM systems any cache is actually\n\u003e\u003e a de-optimization but on low RAM systems it can prevent excessive\n\u003e\u003e paging. This is directly analogous to a CPU cache.\n\u003e \n\u003e \n\u003e I am not entirely sure I agree with that, or understand it\n\u003e correctly.\n\u003e \n\u003e If -for example - the data of some application is a set of\n\u003e records which can be sorted from least frequently used to most\n\u003e frequently used then doing just that sort will beat any\n\u003e application-layer cache. Regardless of size of data and size of\n\u003e RAM, you simply allow the OS to use disk caching or memory map\n\u003e caching to work its magic .\n\nIt's a reasonable assumption, and given that the no-explicit-cache\nimplementation is a subset of the optionally-cached implementation,\nwas of course the initial implementation.\n\n\u003e In fact, I would argue that an application-layer cache *only*\n\u003e makes sense if the data model shows a *hard* distinction between\n\u003e often and not often used data. If usage-frequency is a continuous\n\u003e line, caching is best left to the OS by focussing on proper spatial\n\u003e and temporal locality of reference of your data, because the OS has\n\u003e much more information to make the right decision.\n\nIn practice this is not the case. The Bitcoin data model is neither\ncontinuous nor strictly segregated by usage.\n\nIt is true that with sufficient RAM a cache is totally\ncounterproductive. It is also my experience that an independent UTXO\nstore is not a reasonable/necessary trade of disk space, memory\nscalability, and/or code complexity in exchange for speed.\n\nBut on lower memory systems a explicit cache is beneficial. The\ndifference is clearly measurable in production code by simply changing\nthe cache limit and testing on various configurations.\n\ne\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v2.0.22 (GNU/Linux)\n\niQEcBAEBCAAGBQJY6CXnAAoJEDzYwH8LXOFOf0YH/2qk3hYC6iEDW/DWM2ffkdb9\nQM7A29Pvbfw9Wjr5Xx+ugIQvlAr4T+nByOCT6AnrqNU5K3UUmbC0KIB1rEL94hsK\nQYVlLs0cOrjg8qKJpck+wcgiWw3VbEa/Y44hK7NLUxoy2HsLYaxPhqFH3GGgowqR\nsyga626jf2YUyudZxj1gFuqn7grkwghnzdrEUJMcqQo8IvCqjftGXlKxBGyB/AIs\nDx+5EWO9Q9IxrNpg/fsKKB6xkMxkmSx2hbD7dmEBvi/afbVF66rDTinjInG/LCju\npV7kT/GAWqGQGku6sQyAOexsxVhWA8EA/QEjvbyyGb+3YnR0s6nPk+CxO+RkOgo=\n=e+Pr\n-----END PGP SIGNATURE-----",
"sig": "8deb79f85dead7e51b5f6df0969789eb575b5fee0456029f11ed93b0fd747509ceae31ccf39cb18f09ed2e618565cc80f4bd3e30bc99f58016f1deb4a71dc745"
}