mleku on Nostr: i'm in the midst of formulating a scheme for keeping track of subscriptions on a ...

i'm in the midst of formulating a scheme for keeping track of subscriptions on a relay and ensuring that they don't blow up the memory and garbage collector

one of the questions in my mind is "should there be a timeout for subscriptions that are inactive" and i think the definitive answer is "no"

the proper answer is, maintain a count of how many are open, and send CLOSED for the oldest ones when the number of live subscriptions gets to some certain point

i'm not sure how to evaluate the scheduling or memory use overhead of these exactly, it sorta seems like i should just maybe put simple caps on things like a per connection subscription limit, and send closed if a new one comes in and that limit is exceeded

i printed out NIP-01 to read as i sat at my local sipping whiskey and soda and watching the lizards and the barflies in their ponderances and conversations, and it was nice, i think i shall take it with me to read again a few more times to really soak up what it says and figure out what critical non-stated things are in there, or, i should say, not in there

it seems to me it would be a relatively simple thing to, for example, refuse to open two subscriptions with the identical criteria, so probably i want to derive a hash on subscriptions and use this to prevent this from happening... how to respond to that? probably close the old one and attach the old thread to the new subscription

i also learned quite well that the `limit` in filters only applies to the initial response of sending back matching events... once the relay has sent back that many events, it should then trickle them out, as the client probably doesn't want more than a few each second after it gets the requested limit amount, even if the request way exceeded it, so that's another thing to think about - what should the expected behaviour be? do i send EOSE when the limit expires to indicate this, and then also, perhaps in the in-memory form of the filter, decrement the limit value and once it gets to zero, then it starts to use a limiter that holds it off to some count per few seconds, it could keep decrementing the number, but do a modulo on the number and pause for a second each time it sends the responses? i mean how fast can people use this information, as opposed to a machine, it's really not relevant to ask about the relay serving up raw data as though it was a simple database, that's part of the intent of subscriptions, they are a construct for containing bandwidth for clients that immediately present events to the user

so i'm inclined to think, also, rather than a hard limit on limits, and clamping them, rather, to set a event/time ratio, and feed out events to clients at that maximum rate until the limit passes through zero and then revert to the post-subscription, so prior might be 500/s, and after limit might be 20/s

i don't want to overcomplicate it but i want to make the behaviour as strictly predictable as possible given the inputs to yield a predictable output, and one that is predictable even to a client that may not know there is more coming than they set a limit for

the protocol is very minimal, and flexible, and i am not going to impose rigidity on it but it needs to fit some reasonable limits

when i'm done implementing it i probably will make a couple of suggestions about how to help relay and client devs do nip-01 as cleanly as possible, without any ambiguity or confusion

that is the only part that causes trouble, and the solution isn't always strict rules but reasonable suggestions, usually