girinovey on Nostr: made a little benchmark project: 1- lmdb is really slow to store events. It took 12s ...
made a little benchmark project:
https://github.com/girino/eventstore-benchmark1- lmdb is really slow to store events. It took 12s to store 1000 events. Due to this high preparation time, i tested with only 1000 events (will do bigger tests as soon as lmdb finishes loading all my events).
2- I did not use indexes in slicestore (used the original by fiatjaf).
The table below shows the times for 100 executions of each filter.
My conclusions:
1- Store is orders of magnitude slower in lmdb and this might be the problem i have in raspberry pi. Not querying.
2- non indexed queries are faster in slicestore (limit only and by kind where the index is amost useless)
3- indexed queries are a little faster in lmdb, but not by much. Will probably be slower when i use indexes in memory.
*** This is preliminary data, just for fun, since i have only 1000 events ***
Published at
2024-10-27 18:33:46Event JSON
{
"id": "7b4de57f7bdbd3f9ab83f2b24725edcfd671db2c5809a915d988cf63f70891c3",
"pubkey": "3ffac3a6c859eaaa8cdddb2c7002a6e10b33efeb92d025b14ead6f8a2d656657",
"created_at": 1730054026,
"kind": 1,
"tags": [
[
"e",
"206e8b9aa98f5793524f80df692e335073bb863a1bce3a12d5c08e0e8c886d53",
"wss://nostr.girino.org/",
"root"
],
[
"e",
"01cd042d1208729122f7c2b74bceabe7b5a518df5b7036adc24ab6ec7f0de729",
"",
"reply"
],
[
"p",
"e2ccf7cf20403f3f2a4a55b328f0de3be38558a7d5f33632fdaaefc726c1c8eb",
"",
"mention"
],
[
"client",
"noStrudel",
"31990:266815e0c9210dfa324c6cba3573b14bee49da4209a9456f9484e5106cd408a5:1686066542546"
]
],
"content": "made a little benchmark project: https://github.com/girino/eventstore-benchmark\n1- lmdb is really slow to store events. It took 12s to store 1000 events. Due to this high preparation time, i tested with only 1000 events (will do bigger tests as soon as lmdb finishes loading all my events).\n2- I did not use indexes in slicestore (used the original by fiatjaf). \n\nThe table below shows the times for 100 executions of each filter.\n\nMy conclusions: \n1- Store is orders of magnitude slower in lmdb and this might be the problem i have in raspberry pi. Not querying.\n2- non indexed queries are faster in slicestore (limit only and by kind where the index is amost useless)\n3- indexed queries are a little faster in lmdb, but not by much. Will probably be slower when i use indexes in memory.\n\n*** This is preliminary data, just for fun, since i have only 1000 events ***\n\nhttps://image.nostr.build/d6c24216c57a5edd44610c61f408bcf8d779a95e4fe189a56d206b7f84b18c41.png \n",
"sig": "cd57f4d245a1f0afd24e633c9c9715d8aea701e0c316776a9dda760238fb31221a2acdda126544f7eb24f323b9793269c6b16a26677928604a442e06cd907bd0"
}