One of the downstream effects of the AI boom that I hadn't really thought about is that sites are starting to introduce anti-crawling tech as standard, because they don't want all their bandwidth going to training chatgpt7.
This is going to make life harder for people who run benevolent scrapers. How many sites are going to be blocking the wayback machine now, just because it looks too much like an AI scraperbot?
Foone🏳️⚧️ /
npub135…kgtu7
2024-07-29 20:57:16
Author Public Key
npub135xx0y770mwjtvlxln7jun5mxqwwctr72aafmnczglny6lpuwdlqlkgtu7Published at
2024-07-29 20:57:16Event JSON
{
"id": "e87eff4103f88698590bd34ea91ec98472db109e3c9a28a7e5b6bc015d390b70",
"pubkey": "8d0c6793de7edd25b3e6fcfd2e4e9b301cec2c7e577a9dcf0247e64d7c3c737e",
"created_at": 1722286636,
"kind": 1,
"tags": [
[
"proxy",
"https://digipres.club/users/foone/statuses/112871777024738918",
"activitypub"
]
],
"content": "One of the downstream effects of the AI boom that I hadn't really thought about is that sites are starting to introduce anti-crawling tech as standard, because they don't want all their bandwidth going to training chatgpt7.\n\nThis is going to make life harder for people who run benevolent scrapers. How many sites are going to be blocking the wayback machine now, just because it looks too much like an AI scraperbot?",
"sig": "1f8691072c23449e503678b01248c832866481692dc7d89495c81fb43bb36b7d9f42ccd25a11794c14f2ee30bcca62a5fdaa5080c37a7a3d2005c49dbb3f7bd8"
}