:twiz: (npub1ygr…cg6z) At no point did I actually think to look at their robots.txt… Honestly this is a good ledger of ‘known bad crawlers’ lol.
I found out years ago that (at that time anyways) they had a bunch of regular expressions saved off somewhere to match known spammy/URL shortner URLs so I grabbed those to use to filter my guestbook spam lol
swagg boi :mastoiddong_gold: /
npub1vu…syv2p
2023-08-05 03:54:45
in reply to nevent1q…n4pj
Author Public Key
npub1vuwsup87kt73vczp64mztq3txmkr9mhprlmwk7a9q7fmxzvlgxzqlsyv2pPublished at
2023-08-05 03:54:45Event JSON
{
"id": "adce3c9dd3b02dbff1596f6c22e46b7f3030759b3c77a848333b2c86c58ac44f",
"pubkey": "671d0e04feb2fd166041d57625822b36ec32eee11ff6eb7ba50793b3099f4184",
"created_at": 1691207685,
"kind": 1,
"tags": [
[
"p",
"22069a052fedec4210734322b71360c0bc1eb35081363bb6457ff90c6cbffc04",
"wss://relay.mostr.pub"
],
[
"e",
"aecabdd4226503384329a56fbefe09e1a7774c5cdcf19e5b6c4cf907a2544486",
"wss://relay.mostr.pub",
"reply"
],
[
"proxy",
"https://fedi.seriousbusiness.international/objects/df766d4e-1f74-4262-810f-e8f23ae23f4a",
"activitypub"
]
],
"content": "nostr:npub1ygrf5pf0ahkyyyrngv3twymqcz7pav6ssymrhdj90luscm9llszq56cg6z At no point did I actually think to look at their robots.txt… Honestly this is a good ledger of ‘known bad crawlers’ lol.\n\nI found out years ago that (at that time anyways) they had a bunch of regular expressions saved off somewhere to match known spammy/URL shortner URLs so I grabbed those to use to filter my guestbook spam lol",
"sig": "102955e732d3a79f486cc068de1ac360b91d1c3f90244f1647571a904f9455030b7e32cf2a14614f42113272c77336e3e30c0ba84bb3d4f51cf6c4465d540a57"
}