EXADIO on Nostr: If you don't know which model quantization (the different file name suffixes like ...
If you don't know which model quantization (the different file name suffixes like `Q4_K_M` etc.) to use, follow these steps:
1. Use `Q8_0`
2. Otherwise try `Q6_K`
3. Alternatively `Q4_0` or `IQ4_XS`
Published at
2025-05-22 23:21:34Event JSON
{
"id": "3430b1834f7a3daf9044cc1519d4e81241ca6ed4f7d53b9c381ae406456753ec",
"pubkey": "c9badfeaadbdeccc6e226b3b6d3ff17b132cf3b29095cf117620b2ee610f861a",
"created_at": 1747956094,
"kind": 1,
"tags": [
[
"e",
"99662f3ae80ce37eb3b7b31bb28d621ba162bb4a8b8707d83206b6f146a92445",
"",
"root"
],
[
"p",
"c9badfeaadbdeccc6e226b3b6d3ff17b132cf3b29095cf117620b2ee610f861a"
]
],
"content": "If you don't know which model quantization (the different file name suffixes like `Q4_K_M` etc.) to use, follow these steps:\n\n1. Use `Q8_0`\n2. Otherwise try `Q6_K`\n3. Alternatively `Q4_0` or `IQ4_XS`",
"sig": "a27c0ea8fe5451db3c18c08c18ba360a4987a5a7c05e7803f4b3a6eef509d721eba0e0adb240a95e581411610d2986a0b3012edbffbad770d75d894d6b46acea"
}
