MachuPikacchu on Nostr: If you just want your LLMs to read the content then something like MongoDB would work ...

If you just want your LLMs to read the content then something like MongoDB would work well since it’s a document store. You can add your own metadata.

If you want to query against the data without iterating through the documents each time then you might want to tokenize your content and store in a vector database like Qdrant or similar.