utopiArte on Nostr: Friendica Support #fediAdmin #fediVerse #AI #KI Text for robots.txt to disallow ...

Friendica Support (nprofile…xmvk)
#fediAdmin #fediVerse #AI #KI

Text for robots.txt to disallow access for known AI crawlers:User-Agent: GPTBot
User-Agent: ClaudeBot
User-Agent: Claude-Web
User-Agent: CCBot
User-Agent: Applebot-Extended
User-Agent: Facebookbot
User-Agent: Meta-ExternalAgent
User-Agent: diffbot
User-Agent: PerplexityBot
User-Agent: Omgili
User-Agent: Omgilibot
User-Agent: ImagesiftBot
User-Agent: Bytespider
User-Agent: Amazonbot
User-Agent: Youbot
Disallow: /

robotstxt.com/ai