mleku on Nostr: diggin into the relay building thing and starting to strongly suspect that the ...

diggin into the relay building thing and starting to strongly suspect that the fiatjaf badger event store implementation is the best suspect for the replicatr resource leak bug that shows up after about 24 hours

in any case, i wanted to upgrade the entire codec, and that is done, and benchmarked, and better than everyone else's attempts at it

so, now, it looks like i'm gonna be mostly done on the actual relay logic by this time tomorrow

i can't see how anything is more necessary than simply timing out subscriptions after some reasonable amount of time, but probably it needs more logic like a watcher process that keeps count of how many subscriptions are open and prunes the oldest ones back

even just instrumenting it to see exactly how many open subscriptions there is would help a lot, and i've been digging around in that code, it uses xsync generic concurrent maps so it shouldn't be that hard to have it print out every minute or so a little data on how many subs are open to fully diagnose that problem

anyway, the other thing is that the database code is going 100% single thread, because many already run in parallel, and there's no logic in making it spawn even more threads for nothing

some operations go well in parallel for a single purpose, notably the mark phase for the GC but the sockets already make all that database activity contentious and concurrent so i think in this case slimming the thread count is probably gonna help find the bug if not be the fix to the bug

maybe this week i'll actually achieve what i've been working on for the last 7 weeks

i didn't really have any even vague estimate of what it would take for me to write all the things properly again but with what i learned... nice to know, i'm working at more than 4x efficiency as a go nostr dev now, than i was in the months until june

anyhooo, it comes the time of the sleeps

GN