you could also segment the data set and run multiple queries on it, at a lower cost
i mean, these things need a lot of power to run, most people think it's magic but it's way more expensive than a human expert
human expert needs equal to a couple pints of diesel per day and has a memory system that is holographic so it can be constantly updated
a full text index only costs about 1/8th extra data in the database to implement.
what would probably be the optimal solution is to use a general purpose model to sift through full text search results and highlight the results it considers to be most relevant (as in, semantics, grammar, order)
mleku
npub1fj…mleku
2025-06-12 08:10:21
in reply to nevent1q…gmhl
Author Public Key
npub1fjqqy4a93z5zsjwsfxqhc2764kvykfdyttvldkkkdera8dr78vhsmmlekuPublished at
2025-06-12 08:10:21Event JSON
{
"id": "fe50e18259880439614d7174f80cd982b150cb5b31ed8922550a3781558d6749",
"pubkey": "4c800257a588a82849d049817c2bdaad984b25a45ad9f6dad66e47d3b47e3b2f",
"created_at": 1749715821,
"kind": 1,
"tags": [
[
"e",
"00d8b745e61777b30a030ba9ec63eade52ac2d1a809b92b6fec5ca3883dda1d0",
"wss://nostr.land",
"root",
"dd664d5e4016433a8cd69f005ae1480804351789b59de5af06276de65633d319"
],
[
"e",
"18ad1ded6081101b1f6c2fd4bc0b19a894c784e3c0cfe489f440b308c3d62442",
"wss://relay.primal.net/",
"reply",
"4c800257a588a82849d049817c2bdaad984b25a45ad9f6dad66e47d3b47e3b2f"
],
[
"p",
"b90c3cb71d66343e01104d5c9adf7db05d36653b17601ff9b2eebaa81be67823"
],
[
"p",
"dd664d5e4016433a8cd69f005ae1480804351789b59de5af06276de65633d319"
]
],
"content": "you could also segment the data set and run multiple queries on it, at a lower cost\n\ni mean, these things need a lot of power to run, most people think it's magic but it's way more expensive than a human expert\n\nhuman expert needs equal to a couple pints of diesel per day and has a memory system that is holographic so it can be constantly updated\n\na full text index only costs about 1/8th extra data in the database to implement.\n\nwhat would probably be the optimal solution is to use a general purpose model to sift through full text search results and highlight the results it considers to be most relevant (as in, semantics, grammar, order)",
"sig": "f3a68aceff9862fd82adc5eb20bddf08f6c2bbb92d3fe5401993eef39fa1c6cbe5e157a148ae805929a44e2001d8f633fbb499ea238618dbf42b7f8572b685b4"
}