scsibug

Sorry, this one escaped onto the network, was meant to be local only (and this is a broken example). Reviewing https://github.com/nostr-protocol/nips/pull/635 to see what we should do for escaping control characters other than /n and ", such as the bell character (\b or 0x07).

2023-07-03 16:31:20

This is a test of an event with a bell \b.

2023-05-14 02:11:52

We made it guys, we are on the map! https://anvaka.github.io/map-of-github/#9.34/-29.1725/45.2594

2023-05-06 19:46:08

- reply

If you used my sample docs to setup HAproxy for a relay, check out this patch that ensures Damus requests make it to the correct backend: https://git.sr.ht/~gheartsfield/nostr-rs-relay/commit/1595ec783d1312206c0a1d7c66b5bfa1ff83f3a4

2023-05-06 19:34:32

- reply

Fixed! Minor difference in how Damus sends a Host header (for default ports, damus sends hostname:port, even for default ports) compared to other clients. My HAProxy rules were doing an exact match for the hostname, changed it to a prefix match, and all works!

2023-05-06 17:35:26

Loaner iPad acquired, gonna finally figure out why Damus and wellorder relay can't seem to talk. #[0]

2023-05-04 00:14:22

- reply

What service is hosting that? I am not running on a CDN, but hetzner has free egress (I do 30-50TB/month on my instance for about $40/month).

2023-05-03 23:25:36

- reply

My impression is there are lots of tradeoffs between concurrency, read/write workloads, appends vs updates, compression, query complexity... LMDB is not known for fast writes, as I understand it.

2023-05-03 20:36:46

- reply

Are you sure that one of those pubkeys doesn't have 40 events more recent than the other?

2023-05-03 20:32:06

- reply

I can't duplicate on nostr-rs-relay/sqlite. This query shows results for both pubkeys:
```
["REQ", "lim", {"authors": ["35d26e4690cbe1a898af61cc3515661eb5fa763b57bd0b42e45099c8b32fd50f", "887645fef0ce0c3c1218d2f5d8e6132a19304cdc57cd20281d082f38cfea0072"], "limit": 10}]
```

2023-05-03 20:22:11

- reply

I agree that would be a bug. The query should sort by time, and show the most recent 40 events taking into account both pubkeys and the age of their respective events. I will try and duplicate it myself.

2023-05-02 02:32:38

- reply

lol, those black keys are like AI hands

2023-04-30 19:11:57

Heading back home from bitcoin++; Thanks to all the great speakers, organizers, volunteers, and attendees (best of luck to all for the hackathon) and many new friends. Thanks #[0] for the invite, it was a fantastic experience for me!

2023-04-28 16:51:00

Opened my first lightning channel at btc++ during the lightning LARP!

2023-04-26 02:39:49

- reply

Keeping an eye on this, what a great vision.

2023-04-24 20:35:59

- reply

Wild, just noticed that my webserver was a bit busier than usual!

2023-04-23 21:43:18

- reply

Thanks! Ya, I suppose it would be nice for paid relays to examine their user's profile for lud06/lud16 fields, call out to the endpoint and auto-whitelist the associated keys. Was reading the spec and was wondering how paid relays avoided dropping those events.

2023-04-23 15:26:44

Are paid relays automatically whitelisting pubkeys for lnurl/wallet providers associated with their paid users?

2023-04-22 19:08:06

nostr-rs-relay 0.8.9 out; includes experimental pay-to-relay support, new jemalloc allocator, and other various improvements. https://git.sr.ht/~gheartsfield/nostr-rs-relay/refs/0.8.9

2023-04-21 23:26:45

- reply

My planned safety valve if I need to free up space or trim indexes has always been to cull likes. I hope clients maintain some way of bookmarking/saving posts locally (which is mostly why I use likes).

2023-04-16 23:32:45

- reply

Stole my idea, but with a better name. Love crablang.

2023-04-13 21:15:03

- reply

Is there any real use of NIP-26? I have a total of nine events on my relay with delegation tags. It is like 1 per million events.

2023-04-10 20:19:13

I will be giving a talk on Nostr at the Bitcoin++ conference in Austin TX, end of this month, hoping to meet some other nostr people! https://btcplusplus.dev/talks#nostr

2023-04-07 20:37:11

- reply

No longer believe the TLS theory since I have identified Damus user agents connecting for NIP-11 relay info successfully. Have not found any Damus clients even sending an HTTP upgrade request for a websocket, so I am still in the dark. Will try to get my hands on an iPhone sometime and try myself.

2023-04-05 16:03:48

- reply

I do not know what is broken. I do not have an iPhone to test Damus with.

2023-04-05 02:12:57

- reply

Next time! We will be back for sure.

2023-04-04 19:28:53

- reply

I monitor event rates from IPs, use a bayes filter trained on nostr spam, penalize large events, give credit to nip-05 and other quality signals, lots of small things basically. Not perfect, but good enough for what I see today.

2023-04-04 19:26:46

- reply

Configuring is easy, just point to a gRPC server endpoint. Implementing the spam filter is left as an exercise for the reader... Big opportunity for projects to do stuff here. Mine isn't open source cause it is a bit too hacky to support at the moment (I have some homebrew jobs that analyze events for a bayes model).

2023-04-04 12:59:26

Very nice too that my relay survived basically a month of being ignored, spam filter worked great.

2023-04-04 12:57:35

Back from my honeymoon to Italy! Milan, Tuscany, Florence, and Monza; My first time to visit Europe, an unforgettable and beautiful start to marriage with my amazing partner for life!

2023-03-30 19:44:36

- reply

And for the record, I was an early bitcoin adopter and then skeptic for years, before getting more excited about it after nostr. So please feel free to have your own opinion of any crypto.

2023-03-30 19:43:13

- reply

Nope, spam filtering has been pretty hands off for the past few weeks. Thanks for the excellent writeup!

2023-03-21 00:11:03

Heading out to my honeymoon in Italy shortly. 🫂 to everyone attending nostrica in person or virtually, it looked amazing down there! Be back in April!

2023-03-06 23:46:42

- reply

Added support for these in nostr-rs-relay, so if you run that, you can point to an icon in your config file.

2023-03-04 22:52:22

- reply

Postgres just overwhelmed? Any stats on REQS/sec or concurrent users?

2023-03-03 12:38:12

- reply

I run nostr-pub.wellorder.net and would be honored to care for your notes, as well as other stuff.

2023-03-02 03:31:07

Geez; hit 500 megabits/sec upload on my relay. Y'all clients need to chill. I'm sure there is something interesting happening on twitter or something, I dunno. #MarchOffTwitter

2023-03-02 00:36:15

- reply

Tagging #[3]

2023-03-02 00:35:37

- reply

That one is a 156 KB REQ; I seem to get a ton of them. Hopefully iris.to can look into/fix this... otherwise I may have to break this one :/

2023-03-02 00:33:47

iris.to sending some very strange REQs that send stuff like: "authors":[{},{},{},{},{},{},{....

2023-03-01 13:35:34

- reply

(Add your WoS address into your profile too, so that others can zap it)

2023-03-01 13:34:58

- reply

On Android, it is as simple as installing Wallet Of Satoshi. Amethyst will find it automatically.

2023-03-01 13:32:55

- reply

My notes (and others) from back in 2021 are still on nostr-pub.wellorder.net. You may be seeing relays that only return a limited number of results for a query (1000), plus clients that don't work well with that (just a guess).

2023-02-28 23:32:33

- reply

Unclebob maintains a cool desktop client, more-speech (clojure).

2023-02-28 23:21:17

Thanks for the zaps, #[0]. Really appreciate the support!

2023-02-27 13:50:10

- reply

#2 and #3 are tied for me.

2023-02-27 03:22:14

- reply

Only nostr-pub at the moment, but that will change soon.

2023-02-26 16:46:39

Reading up on timely-dataflow (https://github.com/TimelyDataflow/timely-dataflow) today. Would be awesome to have an incremental pagerank/trustrank running against nostr pubkeys.

2023-02-25 23:38:40

- reply

For my relay, I have higher rate limits for nip-05 pubkeys. It is just one signal among many and reduces false-positive spam hits; I would very much like to keep things open for any non-abusive account, regardless of nip-05. I do monitor the rejected events, and am fairly confident I only kick out spam (as defined by me).

2023-02-25 21:45:47

Just merged pay-to-relay into nostr-rs-relay master branch (great work by #[0]). Totally untested by me, any help there is greatly appreciated!

2023-02-25 19:40:47

- reply

Right now it is a strong signal that you are legit (almost all spam is _not_ NIP-05 verified). If it is from a legit domain, even more so. NIP-05 domain operators have some motivation to not verify spam accounts. I don't expect to see spam being a problem from nostrpurple.com, for example.

2023-02-25 15:47:30

- reply

nostr-pub has spam filtering. nostr-verified does not currently.

2023-02-24 03:55:27

- reply

Nothing public, but I would encourage you to look at both the gRPC API for nostr-rs-relay and the example code in the repo (only a rust sample, but contributions welcome), and the great work at https://spam.nostr.band/ to provide approaches and data. Feel free to DM me with any questions!

2023-02-23 22:49:28

- reply

More than a dozen, but it includes looking at well formed events for the kind, proof of work on pubkey/event, Nip05 verification, event rate from the source IP, a naive bayes spam model, event size, whether it is an event going to the global feed/timeline, and known spammy strings & regexes... etc. All of these get combined to determine a probability of spam, and there is a cutoff to reject the event.

2023-02-23 22:00:54

- reply

I only hope we're having as much fun as the PHP people.

2023-02-23 21:55:44

- reply

Any NIP-05 is a plus. I have a list of 100+NIP-05 domains that have been around long enough to get extra bonus points. If you just post normally, you'll get added to that over time and the spam filter will be even less likely to affect you.

2023-02-23 21:06:02

- reply

For my (free) relay, NIP-05 validation and/or PoW is the best way to ensure your events get persisted. But most non-spammy events make it through just fine.

2023-02-23 21:05:07

- reply

We absolutely are. I'm blocking hundreds of thousands of events a day.

2023-02-21 14:19:27

- reply

Make sure you use release 0.8.8 or newer, I just fixed some issues with the in-memory DB. Thanks for reminding me to test it :)

2023-02-21 13:27:51

- reply

If you want a public relay that is happy to be used for development/testing, I have wss://nostr-dev.wellorder.net just for that.

2023-02-21 13:26:25

- reply

Indeed, nostr-rs-relay supports an in-memory mode (SQLite). Just set database.in_memory=true in the config file.

2023-02-18 23:20:34

- reply

IP blocks; block known spammy strings; bayes spam filters, require PoW, require payment, whitelist pubkeys and their follows, and aggressive rate limiting... or some combo of all the above.

2023-02-18 22:35:09

- reply

Ya, need software details, nostr was meant for terminals!

2023-02-18 22:33:21

- reply

Welcome to Nostr! Shout out if you have any questions.

2023-02-18 03:08:37

- reply

Lol, that didn't last long. 0.8.7 has fixes for a couple embarrassing bugs for logging & postgres SQL.

2023-02-17 20:19:15

nostr-rs-relay 0.8.6 now out; with support for expiring events (NIP-40) as well as authentication (NIP-42). Cheers, and thanks to contributors #[0], #[1], #[2], and rorp!

2023-02-16 03:05:16

- reply

Amazing, thanks for sharing!

2023-02-15 21:59:04

- reply

Noticed this too. Hitting my domains that have zero nostr identities associated.

2023-02-15 19:12:32

- reply

Not in WAL mode: readers and writers can proceed without blocking one another. Checkpoints can be an issue for concurrency, but that can be worked around.

2023-02-15 19:06:55

- reply

Plan is most likely to allow gRPC to deny, permit, or permit w/expiration. Need to implement the expiration NIP first, that is going to be a little tricky to not have a performance regression on.

2023-02-15 19:04:49

- reply

SQLite can use many cores, and even multiple processes. There are some gotchas, but generally speaking it can use every core available for queries (especially reads).

2023-02-15 00:58:43

- reply

Schemas are driven by the DB. Schemas, indexes, and queries need to be tailored to capabilities and strengths of each DB engine.

2023-02-15 00:53:59

- reply

Typescript was a great choice too. Postgres seems to be doing all the heavy lifting.

2023-02-15 00:50:46

- reply

Scaling just fine right now. To be honest, the goal was always to be able to scale /down/, not necessarily up. But latest release has been pushing up to 200mbits/sec while using a single core, so good enough for many of us.

2023-02-15 00:45:00

- reply

Best group of open source devs I have worked with, keeps me coming back each day to make things as good as I am able to. Shout out to all the relay devs either learning rust with me or teaching me!

2023-02-14 23:32:07

- reply

Very cool. What version is packaged?

2023-02-14 19:25:18

- reply

Everything is in testing, go for it. Nostr is nothing if not resilient.

2023-02-13 00:50:34

- reply

Prometheus to scrape /metrics, connect that to grafana, then build your dashboards. DM me if you need help or examples!

2023-02-12 21:55:15

Another major sqlite performance improvement in nostr-rs-relay just pushed to master. Rebuilds tag table so it takes a minute to upgrade, but some of the most expensive queries run 10x faster. feedback from testing welcome!

2023-02-12 03:25:41

- reply

Yes, I think so. No concrete plans yet however. Probably will be more focused on enforcing rate limiting instead of authorizing REQs (although both are useful).

2023-02-11 19:43:58

- reply

Yep! In your language of choice. I made a toy demo server in Rust/tonic, but you could do something in python, go, whatever can support gRPC.

2023-02-11 19:42:09

- reply

It lets relay operators build or plugin text classification, ip lookup/reputation, abuse detection, or whatever other ways they can think of to control what their relay stores. Not really intended to do anything for relay-to-relay.

2023-02-11 19:38:35

A toy example gRPC event admission service that permits events with specific kinds: https://git.sr.ht/~gheartsfield/nostr-rs-relay/tree/master/item/nauthz_server_example/src/main.rs

2023-02-11 19:33:24

- reply

Here is my take on this: https://git.sr.ht/~gheartsfield/nostr-rs-relay/tree/master/item/proto/nauthz.proto

2023-02-11 19:30:40

nostr-rs-relay now supports gRPC for filtering events to your relay! Free relays can now fight back against spam. Still experimental, but check out the schema, and build cool stuff: https://git.sr.ht/~gheartsfield/nostr-rs-relay/tree/master/item/proto/nauthz.proto

2023-02-11 15:48:50

- reply

Absolutely. It would be killer to have support for sqlite, postgres, AND lmdb.

2023-02-11 14:14:57

I'm a lightning noob but will need to start testing some pay-to-relay stuff. Would appreciate any sats to further nostr-rs-relay development!

2023-02-11 11:00:30

- reply

I will post my attempt today, just got it working last night.

2023-02-11 10:59:07

- reply

In my grpc implementation I also provide IP, user agent, origin, and NIP-05 address, if one exists. I map all the event fields into proto fields, for tags I may cheat and make that a serialized json string.

2023-02-11 10:48:21

- reply

There is a PR in work to add pay-to-relay, so probably: https://github.com/scsibug/nostr-rs-relay/pull/64

2023-02-09 21:17:54

- reply

Mostly chatty clients requesting big chunks of the feed over and over. Other relays streaming results to stay in sync is minimal. Check out how much web clients pull down from relays in MB, multiply by hundreds of thousands a day, you get the picture.

2023-02-08 03:59:52

- reply

Right, and adding on more and more spam prevention/detection will imcur other costs, as well as customer support. Thise kinds of value added features will cost far more than running a stock relay I imagine.

2023-02-08 03:56:14

- reply

Clients can be a massive influence on this, and that is probably where most future gains will come from. Strfry (and others) have proven that you can pump out data as fast as you have bandwidth in most cases.

2023-02-08 03:53:37

- reply

nostr-pub.wellorder.net runs on a 6c/12t AMD, 64GB RAM, 2x512TB nvme dedicated server with unmetered gigabit connection. Nostr needs 1-2 cores & ~6GB RAM typically, and I do about a TB/day of traffic. Managing spammers is more important than beefy machines.