Lex Plt on Nostr: If you have a blog (or any website really), block meta-externalagent via robots.txt ...
Published at
2024-10-30 14:17:53Event JSON
{
"id": "57baefcfd70eaea8afbc45c2a36b02dc1e0384f7ab73e389610e5ff2b0a0ae3a",
"pubkey": "93c58ed1322d682f9fb5a663352f29b41719bee2cff1a2a346afdbe7fad8a025",
"created_at": 1730297873,
"kind": 1,
"tags": [
[
"proxy",
"https://hachyderm.io/users/lexplt/statuses/113396801407177563",
"activitypub"
]
],
"content": "If you have a blog (or any website really), block meta-externalagent via robots.txt (or even better via fail2ban filters).\n\nThis thing crawls everything to enhance their AI\n\nhttps://developers.facebook.com/docs/sharing/webmasters/web-crawlers",
"sig": "492c5526a859712e03599d52b3f1438cec11491efe43c052bcd7cb24246d079511ecdb6346cb3ef52b072d42e8237eea249bd8306ed825b0e2a0659814b6f63d"
}