In Go, os.ReadDir cannot read entries incrementally and calls lstat on each entry. Terrible interface if you need to handle arbitrarily large directories. A puzzling choice when POSIX already have the correct API (opendir/readdir_r is incremental and fast).
A tolerable alternative is filepath.WalkDir (Go 1.16+): no more lstat, but it still forces you to read all files in each directory.
Seriously bad design.
Nicolas Martyanoff /
npub1dk…h54dt
2025-02-16 09:39:37
Author Public Key
npub1dkg70vzk4vvfsc5cup9gkfu2tcrs5unysl7npfvegf78nkuswr5skh54dtPublished at
2025-02-16 09:39:37Event JSON
{
"id": "07adbe7d5238db68bd4ba2c73422dcb67ffcf20f2c82e3960f8c37eca3a2287c",
"pubkey": "6d91e7b056ab18986298e04a8b278a5e070a726487fd30a599427c79db9070e9",
"created_at": 1739698777,
"kind": 1,
"tags": [
[
"proxy",
"https://fosstodon.org/users/galdor/statuses/114012899110119456",
"activitypub"
]
],
"content": "In Go, os.ReadDir cannot read entries incrementally and calls lstat on each entry. Terrible interface if you need to handle arbitrarily large directories. A puzzling choice when POSIX already have the correct API (opendir/readdir_r is incremental and fast).\n\nA tolerable alternative is filepath.WalkDir (Go 1.16+): no more lstat, but it still forces you to read all files in each directory.\n\nSeriously bad design.",
"sig": "a582041c47b556200944664ee7dbfe29814a1688d837e41dbb58ebde8919836898f2d28f924a7ae28ec10308b664896aead2bad0fd179e596f4cddf789de89b9"
}