my biggest reason for this change is simple
the json encoder is actually the damned fastest, and hell i'd store the events in the DB directly as json if it were as space efficient as protobuf3... but besides all that, when events are queried, they have to be unmarshaled and then re-marshaled when dispatching to the clients (for queries and when events come in, for subscriptions that match, the matching requires them to be unmarshaled).
i keep saying this but for some reason the hipsters all like CBOR which is a pile of shit - the broadest and most complete binary codec support for all languages is protobuf
i'm of the opinion that some little ways down the track there will be many codecs used because it's very simple to implement to add an encoding header in the http websocket headers, one of the biggest advantages of using websockets... then all the hipsters can make their cbor clients and relays, but until then, it's merely insane to put json in the database for space reasons
oh yeah, and, not to forget... one of the things that is still in my code is the runtime versions of a, e and p tags field 2 are still in a binary form... the filter also is like this, and this nearly doubles the speed of performing a match on them because both the id/pubkey and tag data are in the same format natively
that's the main reason why my json encoder is so fast, it doesn't double-process anything
mleku / mleku ✝Ⓐ☯︎
npub1fj…mleku
2024-11-16 09:37:39
in reply to nevent1q…ah8h
Author Public Key
npub1fjqqy4a93z5zsjwsfxqhc2764kvykfdyttvldkkkdera8dr78vhsmmlekuPublished at
2024-11-16 09:37:39Event JSON
{
"id": "372142e72dfaedfb03b2cbd9b2901e9aea9d63cb6f08207bc79a08706af5d53e",
"pubkey": "4c800257a588a82849d049817c2bdaad984b25a45ad9f6dad66e47d3b47e3b2f",
"created_at": 1731749859,
"kind": 1,
"tags": [
[
"e",
"fb804add080068b4b6695d2b1d3acd31bf2e113998f92b6922500ff295ffb2e6",
"wss://nostr.land/",
"root"
],
[
"e",
"fb804add080068b4b6695d2b1d3acd31bf2e113998f92b6922500ff295ffb2e6",
"wss://nostr.land/",
"reply"
],
[
"client",
"noStrudel",
"31990:266815e0c9210dfa324c6cba3573b14bee49da4209a9456f9484e5106cd408a5:1686066542546"
]
],
"content": "my biggest reason for this change is simple\n\nthe json encoder is actually the damned fastest, and hell i'd store the events in the DB directly as json if it were as space efficient as protobuf3... but besides all that, when events are queried, they have to be unmarshaled and then re-marshaled when dispatching to the clients (for queries and when events come in, for subscriptions that match, the matching requires them to be unmarshaled).\n\ni keep saying this but for some reason the hipsters all like CBOR which is a pile of shit - the broadest and most complete binary codec support for all languages is protobuf\n\ni'm of the opinion that some little ways down the track there will be many codecs used because it's very simple to implement to add an encoding header in the http websocket headers, one of the biggest advantages of using websockets... then all the hipsters can make their cbor clients and relays, but until then, it's merely insane to put json in the database for space reasons\n\noh yeah, and, not to forget... one of the things that is still in my code is the runtime versions of a, e and p tags field 2 are still in a binary form... the filter also is like this, and this nearly doubles the speed of performing a match on them because both the id/pubkey and tag data are in the same format natively\n\nthat's the main reason why my json encoder is so fast, it doesn't double-process anything",
"sig": "5178ff8a180354bf836e5be0279b48b742d9cff8cba1a45dc5fd07b0f90ec0ae161b12318d61a6291c58086260fb03917d2b8e40b616bef5427688184ea10b68"
}