Yesterday was a rough day for me as a #sysadmin, but #ceph still maintained its perfect record of no data loss, even though I don't 💯 understand all the inner workings.
At this point, I'd say it is battle tested. I've had concurrent drive failures, and this latest issue where it couldn't find the DB device fors of 17 of the 21 drives!
I'm not sure I'd go with ceph again if I had to start over. It allows live migrating running VMs to anither host, which is a huge win for avoiding downtime and saving disk writes for migrating an entire hard drive (as opposed to just migrating the confents of memory). But having standalone disks would be way more performent in terms of both I/O and memory usage.
If I could get equal disk I/O performance, I'd choose ceph for any #SelfHosting. Maybe not for a #homelab, where nobody relies on any of the services, but certainly if you're using it in your daily life.
I still have a #bounty out of 300K sats for anyone who can get me 100MB/s random read inside a VM and 100 IOPS for the rados write benchmark.
What I actually want is for VMs to run at least as well as if they were on a standalone consumer grade hard drive, but I'm trying to provide concrete metrics so there's an objective metric of whether it goes "fast enough".
Dr. Hax
npub16v…meqha
2024-10-06 19:56:25
Author Public Key
npub16v82nr4xt62nlydtj0mtxr49r6enc5r0sl2f7cq2zwdw7q92j5gs8meqhaPublished at
2024-10-06 19:56:25Event JSON
{
"id": "f5aa582e74dae3aa06d535bfa8d1bd005a3998ed43019a18bb6ec9ee552e6f17",
"pubkey": "d30ea98ea65e953f91ab93f6b30ea51eb33c506f87d49f600a139aef00aa9511",
"created_at": 1728244585,
"kind": 1,
"tags": [
[
"t",
"sysadmin"
],
[
"t",
"ceph"
],
[
"t",
"SelfHosting"
],
[
"t",
"selfhosting"
],
[
"t",
"homelab"
],
[
"t",
"bounty"
]
],
"content": "Yesterday was a rough day for me as a #sysadmin, but #ceph still maintained its perfect record of no data loss, even though I don't 💯 understand all the inner workings.\n\nAt this point, I'd say it is battle tested. I've had concurrent drive failures, and this latest issue where it couldn't find the DB device fors of 17 of the 21 drives!\n\nI'm not sure I'd go with ceph again if I had to start over. It allows live migrating running VMs to anither host, which is a huge win for avoiding downtime and saving disk writes for migrating an entire hard drive (as opposed to just migrating the confents of memory). But having standalone disks would be way more performent in terms of both I/O and memory usage.\n\nIf I could get equal disk I/O performance, I'd choose ceph for any #SelfHosting. Maybe not for a #homelab, where nobody relies on any of the services, but certainly if you're using it in your daily life.\n\nI still have a #bounty out of 300K sats for anyone who can get me 100MB/s random read inside a VM and 100 IOPS for the rados write benchmark.\n\nWhat I actually want is for VMs to run at least as well as if they were on a standalone consumer grade hard drive, but I'm trying to provide concrete metrics so there's an objective metric of whether it goes \"fast enough\".",
"sig": "2e028f1d5242935db4f4df16d0a54fe3124ef08867f9aa04bff3d9026a8d5857164d28f7e9bb6a8f0b5cbcc15b4c3eada2e15eaaea4a50bb57253aedb84c386a"
}