We investigated performance degradation due to excessive crawling of Codeberg.
We often trace back such things to company websites that proudly list their key investors - but don't actually explain what they are doing (or why they need to crawl Codeberg more excessively than the crawlers of the major search engines combined).
It's probably not a use case we believe in anyway, so we blocked several large IP ranges today.
Codeberg.org /
npub1fh…3ee83
2024-06-03 12:50:30
Author Public Key
npub1fh736ae0xzk98s59wggazdpqg0g78mg2rwk7qf87ew96cqhc4lrst3ee83Published at
2024-06-03 12:50:30Event JSON
{
"id": "e79b8e8514b196f95f6092ecc38a550e63a669a18583c27979914504b6fa1809",
"pubkey": "4dfd1d772f30ac53c2857211d1342043d1e3ed0a1bade024fecb8bac02f8afc7",
"created_at": 1717419030,
"kind": 1,
"tags": [
[
"proxy",
"https://social.anoxinon.de/@Codeberg/112552773571280070",
"web"
],
[
"proxy",
"https://social.anoxinon.de/users/Codeberg/statuses/112552773571280070",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://social.anoxinon.de/users/Codeberg/statuses/112552773571280070",
"pink.momostr"
]
],
"content": "We investigated performance degradation due to excessive crawling of Codeberg.\n\nWe often trace back such things to company websites that proudly list their key investors - but don't actually explain what they are doing (or why they need to crawl Codeberg more excessively than the crawlers of the major search engines combined).\n\nIt's probably not a use case we believe in anyway, so we blocked several large IP ranges today.",
"sig": "784410092ce96f08fb2c0ba605eac65a13dc9048135fcbae0483f09f1debde255730d82964a3a2c9cd8bc9199a3d416061267cc5d4d7584e85c32e01b06dff56"
}