#realy #devstr #progressrepot
trying to sleep this evening i got thinking a lot about the next task i have set for myself, which is designing the database indexes
this document contains my first draft of all of the indexes that will be created for the https://x.realy.lol and all of the features it will have are related to this. i started to understand how fiatjaf had designed the indexes for his badger event store, and had started to extend them with https://realy.lol but this is a complete set including a new key prefix scheme, and this will enable all kinds of searches including a full text inverted index search, and required access time/frequency data that will enable configurable pruning scheme for expiring events from the store
it also will enable other advanced capabilities, such as indexing other than the simple single letter tags - and, i am forgetting - a special index that contains a set of several complete metadata fields for returning only the event ID, excluding by pubkey, filtering by kind and timestamp
several of these are beyond the specification but enable features that will be part of the full HTTP API, which also will include the ability to search and retrieve ranges of events based on their database sequence number (serial) which facilitates fast periodic syncing by scrapers who can simply maintain the last highest serial number to know where to pick up from
mleku
npub1fj…mleku
2025-05-31 04:00:21
Author Public Key
npub1fjqqy4a93z5zsjwsfxqhc2764kvykfdyttvldkkkdera8dr78vhsmmlekuPublished at
2025-05-31 04:00:21Event JSON
{
"id": "1bc0de36aa88dbfe985dd29c03098963e71aa0535abbea10701ca480514d85b3",
"pubkey": "4c800257a588a82849d049817c2bdaad984b25a45ad9f6dad66e47d3b47e3b2f",
"created_at": 1748664021,
"kind": 1,
"tags": [
[
"t",
"realy"
],
[
"t",
"devstr"
],
[
"t",
"progressrepot"
],
[
"client",
"noStrudel",
"31990:266815e0c9210dfa324c6cba3573b14bee49da4209a9456f9484e5106cd408a5:1686066542546"
]
],
"content": "#realy #devstr #progressrepot\n\ntrying to sleep this evening i got thinking a lot about the next task i have set for myself, which is designing the database indexes\n\nthis document contains my first draft of all of the indexes that will be created for the https://x.realy.lol and all of the features it will have are related to this. i started to understand how fiatjaf had designed the indexes for his badger event store, and had started to extend them with https://realy.lol but this is a complete set including a new key prefix scheme, and this will enable all kinds of searches including a full text inverted index search, and required access time/frequency data that will enable configurable pruning scheme for expiring events from the store\n\nit also will enable other advanced capabilities, such as indexing other than the simple single letter tags - and, i am forgetting - a special index that contains a set of several complete metadata fields for returning only the event ID, excluding by pubkey, filtering by kind and timestamp\n\nseveral of these are beyond the specification but enable features that will be part of the full HTTP API, which also will include the ability to search and retrieve ranges of events based on their database sequence number (serial) which facilitates fast periodic syncing by scrapers who can simply maintain the last highest serial number to know where to pick up from",
"sig": "ed12f1392b828054fe63af4578b1a1415ea24697310ea062c4a3ed9c368e4c4f1f00a52f975e8ca28565f18ce96929fb8c0486d8b6e741419fb1dc9a36739741"
}