How to let the humans read our website again.
$ vnstat -h
... 2.04 Mbit/s ...
$ echo "User-agent: GPTBot\nDisallow: /" >> /var/www/robots.txt
... https://openai.com/gptbot-ranges.txt (open in browser, because lol, *they* block wget and curl)
$ iptables -A INPUT -s 52.230.152.0/24 -j DROP
$ iptables -A INPUT -s 52.233.106.0/24 -j DROP
$ iptables -A INPUT -s 20.171.206.0/24 -j DROP
$ vnstat -h
... 514.16 kbit/s ...
🤷
#openai #crawler #bullshit
Malvin Gattinger /
npub1em…yvgqe
2024-10-14 10:26:13
Author Public Key
npub1emjy3elq9srpv8w0pnungr05pmp0cm58drkhpywpuv9gs6xrv46s2yvgqePublished at
2024-10-14 10:26:13Event JSON
{
"id": "5c9c756d74818217927c9c7bf2afd5565e6663073df7609f11e75dfe2dd6cf9d",
"pubkey": "cee448e7e02c06161dcf0cf9340df40ec2fc6e8768ed7091c1e30a8868c36575",
"created_at": 1728901573,
"kind": 1,
"tags": [
[
"t",
"openai"
],
[
"t",
"crawler"
],
[
"t",
"bullshit"
],
[
"proxy",
"https://scholar.social/users/m4lvin/statuses/113305293508463296",
"activitypub"
]
],
"content": "How to let the humans read our website again.\n\n$ vnstat -h\n... 2.04 Mbit/s ...\n\n$ echo \"User-agent: GPTBot\\nDisallow: /\" \u003e\u003e /var/www/robots.txt\n\n... https://openai.com/gptbot-ranges.txt (open in browser, because lol, *they* block wget and curl)\n\n$ iptables -A INPUT -s 52.230.152.0/24 -j DROP\n$ iptables -A INPUT -s 52.233.106.0/24 -j DROP\n$ iptables -A INPUT -s 20.171.206.0/24 -j DROP\n\n$ vnstat -h\n... 514.16 kbit/s ...\n\n🤷 \n\n#openai #crawler #bullshit",
"sig": "4845d5c02a10d4eb8d801436ae20edc125fb2f34b899cda658498aeb8f39b9fae034bde784c5ef0d0e403a029ee21b5440649acd341b317b650de2aa8975d434"
}