scsibug

author of nostr-rs-relay, operates nostr-pub.wellorder.net

Public Key

npub1xhfxu35se0s63x90v8xr29txr66l5a3m277skshy2zvu3ve0658sla4xw3

NIP-05 Address

scsibug@wellorder.net

NIP-57 Address

outsideice77@walletofsatoshi.com

Profile Code

nprofile1qqsrt5nwg6gvhcdgnzhkrnp4z4npad06wca400gtgtj9pxwgkvha2rcpzemhxue69uhhyetvv9ujumn0wd68ytnzv9hxgqgdwaehxw309ahx7uewd3hkck62q5x

Publishing to

relay.nostr.band

nos.lol

Author Public Key

npub1xhfxu35se0s63x90v8xr29txr66l5a3m277skshy2zvu3ve0658sla4xw3

Show more details

Last Notes

2023-09-15 23:05:25

- reply

Lol, my relay has reliably exceeded 30TB/month for all of 2023. It is a bit wild and inefficient.

2023-08-10 19:49:39

- reply

I've got a black lightning beanie and a macbook, CTF'ing near the DJ stand.

2023-08-10 19:27:33

For anyone at #DefCon I will be in the SomaFM chill-out lounge for the next hour. Anybody who wants to meet up and grab lunch or talk, send me a DM!

2023-08-08 23:09:53

- reply

Let's do it. Maybe grabbing lunch tomorrow after getting badges?

2023-08-05 11:31:51

Anybody going to be at #bsidesLV or #defcon next week? I will be at both, would be great to have an informal Nostr meetup for anyone in LV.

2023-07-03 16:38:34

- reply

Sorry, this one escaped onto the network, was meant to be local only (and this is a broken example). Reviewing https://github.com/nostr-protocol/nips/pull/635 to see what we should do for escaping control characters other than /n and ", such as the bell character (\b or 0x07).

2023-07-03 16:31:20

This is a test of an event with a bell \b.

2023-05-14 02:11:52

We made it guys, we are on the map! https://anvaka.github.io/map-of-github/#9.34/-29.1725/45.2594

2023-05-06 19:46:08

- reply

If you used my sample docs to setup HAproxy for a relay, check out this patch that ensures Damus requests make it to the correct backend: https://git.sr.ht/~gheartsfield/nostr-rs-relay/commit/1595ec783d1312206c0a1d7c66b5bfa1ff83f3a4

2023-05-06 19:34:32

- reply

Fixed! Minor difference in how Damus sends a Host header (for default ports, damus sends hostname:port, even for default ports) compared to other clients. My HAProxy rules were doing an exact match for the hostname, changed it to a prefix match, and all works!

2023-05-06 17:35:26

Loaner iPad acquired, gonna finally figure out why Damus and wellorder relay can't seem to talk. #[0]

2023-05-04 00:14:22

- reply

What service is hosting that? I am not running on a CDN, but hetzner has free egress (I do 30-50TB/month on my instance for about $40/month).

2023-05-03 23:25:36

- reply

My impression is there are lots of tradeoffs between concurrency, read/write workloads, appends vs updates, compression, query complexity... LMDB is not known for fast writes, as I understand it.

2023-05-03 20:36:46

- reply

Are you sure that one of those pubkeys doesn't have 40 events more recent than the other?

2023-05-03 20:32:06

- reply

I can't duplicate on nostr-rs-relay/sqlite. This query shows results for both pubkeys:
```
["REQ", "lim", {"authors": ["35d26e4690cbe1a898af61cc3515661eb5fa763b57bd0b42e45099c8b32fd50f", "887645fef0ce0c3c1218d2f5d8e6132a19304cdc57cd20281d082f38cfea0072"], "limit": 10}]
```

2023-05-03 20:22:11

- reply

I agree that would be a bug. The query should sort by time, and show the most recent 40 events taking into account both pubkeys and the age of their respective events. I will try and duplicate it myself.

2023-05-02 02:32:38

- reply

lol, those black keys are like AI hands

2023-04-30 19:11:57

Heading back home from bitcoin++; Thanks to all the great speakers, organizers, volunteers, and attendees (best of luck to all for the hackathon) and many new friends. Thanks #[0] for the invite, it was a fantastic experience for me!

2023-04-26 02:39:49

- reply

Keeping an eye on this, what a great vision.

2023-04-24 20:35:59

- reply

Wild, just noticed that my webserver was a bit busier than usual!

2023-04-23 21:43:18

- reply

Thanks! Ya, I suppose it would be nice for paid relays to examine their user's profile for lud06/lud16 fields, call out to the endpoint and auto-whitelist the associated keys. Was reading the spec and was wondering how paid relays avoided dropping those events.

2023-04-23 15:26:44

Are paid relays automatically whitelisting pubkeys for lnurl/wallet providers associated with their paid users?

2023-04-22 19:08:06

nostr-rs-relay 0.8.9 out; includes experimental pay-to-relay support, new jemalloc allocator, and other various improvements. https://git.sr.ht/~gheartsfield/nostr-rs-relay/refs/0.8.9

2023-04-13 21:15:03

- reply

Is there any real use of NIP-26? I have a total of nine events on my relay with delegation tags. It is like 1 per million events.

2023-04-10 20:19:13

I will be giving a talk on Nostr at the Bitcoin++ conference in Austin TX, end of this month, hoping to meet some other nostr people! https://btcplusplus.dev/talks#nostr

2023-04-07 20:37:11

- reply

No longer believe the TLS theory since I have identified Damus user agents connecting for NIP-11 relay info successfully. Have not found any Damus clients even sending an HTTP upgrade request for a websocket, so I am still in the dark. Will try to get my hands on an iPhone sometime and try myself.

2023-04-04 19:26:46

- reply

Configuring is easy, just point to a gRPC server endpoint. Implementing the spam filter is left as an exercise for the reader... Big opportunity for projects to do stuff here. Mine isn't open source cause it is a bit too hacky to support at the moment (I have some homebrew jobs that analyze events for a bayes model).

2023-04-04 12:59:26

Very nice too that my relay survived basically a month of being ignored, spam filter worked great.

2023-04-04 12:57:35

Back from my honeymoon to Italy! Milan, Tuscany, Florence, and Monza; My first time to visit Europe, an unforgettable and beautiful start to marriage with my amazing partner for life!

2023-03-30 19:43:13

- reply

Nope, spam filtering has been pretty hands off for the past few weeks. Thanks for the excellent writeup!

2023-03-06 23:46:42

- reply

Added support for these in nostr-rs-relay, so if you run that, you can point to an icon in your config file.

2023-03-02 03:31:07

Geez; hit 500 megabits/sec upload on my relay. Y'all clients need to chill. I'm sure there is something interesting happening on twitter or something, I dunno. #MarchOffTwitter

2023-03-02 00:33:47

iris.to sending some very strange REQs that send stuff like: "authors":[{},{},{},{},{},{},{....

2023-03-01 13:35:34

- reply

(Add your WoS address into your profile too, so that others can zap it)

2023-03-01 13:32:55

- reply

My notes (and others) from back in 2021 are still on nostr-pub.wellorder.net. You may be seeing relays that only return a limited number of results for a query (1000), plus clients that don't work well with that (just a guess).

2023-02-28 23:21:17

Thanks for the zaps, #[0]. Really appreciate the support!

2023-02-26 16:46:39

Reading up on timely-dataflow (https://github.com/TimelyDataflow/timely-dataflow) today. Would be awesome to have an incremental pagerank/trustrank running against nostr pubkeys.

2023-02-25 23:38:40

- reply

For my relay, I have higher rate limits for nip-05 pubkeys. It is just one signal among many and reduces false-positive spam hits; I would very much like to keep things open for any non-abusive account, regardless of nip-05. I do monitor the rejected events, and am fairly confident I only kick out spam (as defined by me).

2023-02-25 21:45:47

Just merged pay-to-relay into nostr-rs-relay master branch (great work by #[0]). Totally untested by me, any help there is greatly appreciated!

2023-02-23 21:55:44

- reply

Any NIP-05 is a plus. I have a list of 100+NIP-05 domains that have been around long enough to get extra bonus points. If you just post normally, you'll get added to that over time and the spam filter will be even less likely to affect you.

2023-02-23 21:06:02

- reply

For my (free) relay, NIP-05 validation and/or PoW is the best way to ensure your events get persisted. But most non-spammy events make it through just fine.

2023-02-23 21:05:07

- reply

We absolutely are. I'm blocking hundreds of thousands of events a day.

2023-02-21 14:19:27

- reply

Make sure you use release 0.8.8 or newer, I just fixed some issues with the in-memory DB. Thanks for reminding me to test it :)

2023-02-21 13:27:51

- reply

If you want a public relay that is happy to be used for development/testing, I have wss://nostr-dev.wellorder.net just for that.

2023-02-21 13:26:25

- reply

Indeed, nostr-rs-relay supports an in-memory mode (SQLite). Just set database.in_memory=true in the config file.

2023-02-18 23:20:34

- reply

IP blocks; block known spammy strings; bayes spam filters, require PoW, require payment, whitelist pubkeys and their follows, and aggressive rate limiting... or some combo of all the above.

2023-02-18 22:33:21

- reply

Welcome to Nostr! Shout out if you have any questions.

2023-02-18 03:08:37

- reply

Lol, that didn't last long. 0.8.7 has fixes for a couple embarrassing bugs for logging & postgres SQL.

2023-02-17 20:19:15

nostr-rs-relay 0.8.6 now out; with support for expiring events (NIP-40) as well as authentication (NIP-42). Cheers, and thanks to contributors #[0], #[1], #[2], and rorp!

2023-02-15 21:59:04

- reply

Noticed this too. Hitting my domains that have zero nostr identities associated.

2023-02-15 19:06:55

- reply

Plan is most likely to allow gRPC to deny, permit, or permit w/expiration. Need to implement the expiration NIP first, that is going to be a little tricky to not have a performance regression on.

2023-02-15 00:58:43

- reply

Schemas are driven by the DB. Schemas, indexes, and queries need to be tailored to capabilities and strengths of each DB engine.

2023-02-15 00:45:00

- reply

Best group of open source devs I have worked with, keeps me coming back each day to make things as good as I am able to. Shout out to all the relay devs either learning rust with me or teaching me!

2023-02-14 23:32:07

- reply

Very cool. What version is packaged?

2023-02-14 19:25:18

- reply

Everything is in testing, go for it. Nostr is nothing if not resilient.

2023-02-13 00:50:34

- reply

Prometheus to scrape /metrics, connect that to grafana, then build your dashboards. DM me if you need help or examples!

2023-02-12 21:55:15

Another major sqlite performance improvement in nostr-rs-relay just pushed to master. Rebuilds tag table so it takes a minute to upgrade, but some of the most expensive queries run 10x faster. feedback from testing welcome!

2023-02-11 19:43:58

- reply

Yep! In your language of choice. I made a toy demo server in Rust/tonic, but you could do something in python, go, whatever can support gRPC.

2023-02-11 19:38:35

A toy example gRPC event admission service that permits events with specific kinds: https://git.sr.ht/~gheartsfield/nostr-rs-relay/tree/master/item/nauthz_server_example/src/main.rs

2023-02-11 19:33:24

- reply

Here is my take on this: https://git.sr.ht/~gheartsfield/nostr-rs-relay/tree/master/item/proto/nauthz.proto

2023-02-11 19:30:40

nostr-rs-relay now supports gRPC for filtering events to your relay! Free relays can now fight back against spam. Still experimental, but check out the schema, and build cool stuff: https://git.sr.ht/~gheartsfield/nostr-rs-relay/tree/master/item/proto/nauthz.proto

2023-02-11 15:48:50

- reply

Absolutely. It would be killer to have support for sqlite, postgres, AND lmdb.

2023-02-11 14:14:57

I'm a lightning noob but will need to start testing some pay-to-relay stuff. Would appreciate any sats to further nostr-rs-relay development!

2023-02-11 10:48:21

- reply

There is a PR in work to add pay-to-relay, so probably: https://github.com/scsibug/nostr-rs-relay/pull/64

2023-02-08 03:53:37

- reply

nostr-pub.wellorder.net runs on a 6c/12t AMD, 64GB RAM, 2x512TB nvme dedicated server with unmetered gigabit connection. Nostr needs 1-2 cores & ~6GB RAM typically, and I do about a TB/day of traffic. Managing spammers is more important than beefy machines.

2023-02-08 03:35:07

- reply

My experience is that CPU is THE bottleneck. Assuming your bandwidth costs are unmetered; which is why I moved to hetzner.

2023-02-08 03:24:44

- reply

I ended up hard coding index selection based on the filter contents (indexed by). Did wonders. Not disimilar from the strfry approach of doing the same.

2023-02-07 12:13:08

- reply

I'm up for easy challenges :)

2023-02-04 00:08:04

- reply

I'm not sure that paging is going to be a huge win, the average REQ returns 50 results.

2023-02-04 00:00:45

- reply

Probably on the order of 2-300 REQs sent per second.

2023-02-03 23:39:56

- reply

correct, all kinds. I'll run out of steam from queries way before event creation is a problem.

2023-02-03 23:31:58

- reply

I'll go; nostr-pub.wellorder.net; had 8k concurrent users yesterday (just bumped up the limit). Averaging 6-8 events/second.

2023-02-03 18:56:07

- reply

That is going to mean you haven't given the container read/write access to the directory with your database. If using podman and rootless containers, that is done with unshare. Otherwise you need to set permissions correctly. DM me if that doesn't make sense.

2023-02-02 17:14:41

Just finished "UNIX A History and a Memoir" by Brian Kernighan. Lots of great stories about early computing history and Bell Labs. Well written, and a good info to the UNIX philosophy; recommended.

2023-02-01 16:04:12

- reply

If more relays mean fewer clients connecting to relays, sure. But I think there are a small number of relays that get 90% of the traffic due to being included in clients by default.

2023-01-25 22:48:59

- reply

Thanks for writing this down. I will need to think through pagination a bit more before implementing, but this is very helpful.

2023-01-24 03:51:15

- reply

That will be key to supporting Very Large numbers of clients that stay connected for a long time, for sure!

2023-01-22 00:35:34

- reply

Biased, but the latest 0.7.17 nostr-rs-relay is consuming <1 core and over 1k connections just fine, so I am pretty happy with it. (nostr-pub.wellorder.net)

2023-01-21 16:33:57

- reply

This would be wildly cool to see.

2023-01-21 00:36:39

- reply

For an example, see: https://github.com/skerkour/kerkour.com/issues/1

2023-01-20 21:46:23

Big milestone that so much conversation has shifted from Telegram to Nostr in the past few weeks.

2023-01-20 17:19:37

A June date for nostrica 2023 would be amazing. Will be on honeymoon in late March!

2023-01-20 13:43:32

- reply

Fantastic to hear, thanks!

2023-01-19 20:04:11

Testing photos.
https://live.staticflickr.com/65535/52628715407_b7a89747c4_b.jpg

2023-01-17 13:08:37

- reply

Awesome!

2023-01-16 23:24:25

- reply

Fixed! https://todo.sr.ht/~gheartsfield/nostr-rs-relay/64

2023-01-16 22:44:21

- reply

Good find. Will open a bug for that. Thanks for investigating.

2023-01-16 22:17:01

- reply

Or you can YOLO it and try "chmod 777 data".

2023-01-16 22:16:23

- reply

Directions on https://sr.ht/~gheartsfield/nostr-rs-relay/ demonstrate how to make a mounted directory writeable w/ podman.

2023-01-16 22:15:18

- reply

Sounds like the container can't write to your data folder on the host.

2023-01-15 22:53:46

- reply

Is that a dealbreaker for you? I can tell you how to avoid that if so. But even if you don't wipe them out at first, any updates to profiles/contacts will delete instead of hide the old ones...

2023-01-15 22:11:49

- reply

This is the way; beautiful.

2023-01-15 00:35:23

- reply

Events are being sent from clients, and from scripts that distribute events across relays. So even if you send an event to a single relay, there is a good chance it will end up on multiple relays somewhat quickly. No spec for how this is coordinated though.

2023-01-07 16:11:58

New 1-million-event Nostr dataset for relay bootstrapping, benchmarking, or analytics. https://wiki.wellorder.net/wiki/nostr-datasets/#1m-v1

2023-01-06 19:22:20

@5e7ae588d7d11eac4c25906e6da807e68c6498f49a38e4692be5a089616ceb18 Verifying My Public Key: "scsibug"

2023-01-05 02:14:34

- reply

The next big improvement will be automatically pruning hidden events. In the meantime you can run the commands here from cron: https://git.sr.ht/~gheartsfield/nostr-rs-relay/tree/master/item/docs/database-maintenance.md

2023-01-05 00:23:34

- reply

Lots of queries for 5000 events (with limit), not too many that return 10k+. But lots of repetitive ones that just eat bandwidth. I have some simple protections, but could use more.

2023-01-04 23:57:21

- reply

It's getting better. 0.7.16 has significant performance increases, some common astral queries are 7x faster from my benchmarks.

2023-01-04 23:55:53

- reply

I have a hard cap at 900, which I bump against. And yes, you can get the entire event stream with a subscription, there are no limits other than what the client requests.

2023-01-04 23:50:24

- reply

No worries; nostream is awesome!