mleku on Nostr: my binary codec already does a lot of that memory remapping of fields, as the runtime ...

my binary codec already does a lot of that memory remapping of fields, as the runtime and database versions of the data are basically the same - it keeps the id/pubkey/signature fields (including in common tags) in raw binary and unpacking it into the runtime format is just a matter of creating pointers

the hex encoding is also done with a SIMD hex encoder, and the sha256 hash is done also with a threaded worker based single instruction multiple data so, on avx512 and avx2 that means it runs 2 hashes per CPU thread

switching the binary fields to be kept as binary except up to the wire has a massive benefit

it is so nearly close to being a perfectly servicable wire codec as well, i just didn't design an envelope encoding or binary serialisation for the filters

but other languages probably won't support this kind of optimization very well certainly not javascript

i don't get how javascript parsing is really much slower working in native json (which should be optimized to the max) versus making javascript work with foreign binary data formats for these binary fields

but totally understand why it's hard to make clients that aren't JS/DOM/HTML/CSS based... the whole tech industry has focused on this universal platform and its abomination of a scripting language to the expense of serious languages supporting native apps, and the total lack of adequate multi-platform targeting and adequately broad language support for cocoa, win32, gtk and qt - ironically most of the time it's either electron, putting the front end in a browser engine, or some kind of simple and fairly inadequate immediate mode or similar direct opengl/vulkan based thing (eg imgui, egui, gio, fyne, nucular)