Angle on Nostr: ...So I was looking at the robots.txt file for anticapitalist.party, and it looks ...
...So I was looking at the robots.txt file for anticapitalist.party, and it looks like it explicitly allows gpt to scrape anything? Is that correct? Seems weird... :/
User-agent: GPTBot
Disallow: /
Published at
2024-08-23 21:11:59Event JSON
{
"id": "6904998986f98823dca50599a161e093ea055cfe0150d831f291b9f681f97a22",
"pubkey": "1174fe9082f688d1c328fb9f544fa866152b66357763023f2c8d345c066119aa",
"created_at": 1724447519,
"kind": 1,
"tags": [
[
"proxy",
"https://anticapitalist.party/users/Angle/statuses/113013392614681006",
"activitypub"
]
],
"content": "...So I was looking at the robots.txt file for anticapitalist.party, and it looks like it explicitly allows gpt to scrape anything? Is that correct? Seems weird... :/\n\nUser-agent: GPTBot\nDisallow: /",
"sig": "f35105d3aaf31a444eb19cc82a664a726ca44ee69e8ea85a32b4a011d682c4407818d0fc2bd9de7cce14bd8b002e73e2650e81e90b798b79b986e746da86d3e7"
}
