Yann Büchau :python: on Nostr: :gitannex: #gitAnnex's built-in file deduplicationis is not only useful to save disk ...
#gitAnnex's built-in file deduplicationis is not only useful to save disk space, but also to catch certain errors in your data pipeline:
If after several days of data processing and producing new files from others, git-annex says everything is already synced up, it's an indication that you messed up - like hard-coding the input file path in the script 🤦 😑 😅
Published at
2024-03-18 08:07:08Event JSON
{
"id": "8fc5c9d0f4fa8c345ea62d596db8bcd99371cec0efcfead48917bcf79001d2bb",
"pubkey": "0abc897a05eca0849f658dc45fb983e46041d357150b09df857131e7a7552848",
"created_at": 1710749228,
"kind": 1,
"tags": [
[
"t",
"gitannex"
],
[
"emoji",
"gitannex",
"https://cdn.fosstodon.org/custom_emojis/images/000/783/010/original/3528483de125873c.png"
],
[
"proxy",
"https://fosstodon.org/users/nobodyinperson/statuses/112115661441769505",
"activitypub"
]
],
"content": ":gitannex: #gitAnnex's built-in file deduplicationis is not only useful to save disk space, but also to catch certain errors in your data pipeline:\n\nIf after several days of data processing and producing new files from others, git-annex says everything is already synced up, it's an indication that you messed up - like hard-coding the input file path in the script 🤦 😑 😅",
"sig": "c469fcf36d6542bdbce339c3d6f3e2ec9880cb987d735b269b9ee0f188282c4f998ef012202152ba498482fdc92ec72b572c1222dc8e3278d9188d923a75a374"
}
