gary on Nostr: Currently writing an implementation of the 1-bit quantization linear layer from this ...
Currently writing an implementation of the 1-bit quantization linear layer from this paper. Fresh trained LLM costs go to ~zero if this works at scale.
https://arxiv.org/pdf/2402.17764.pdfPublished at
2024-02-29 23:02:55Event JSON
{
"id": "581fa113267435f5f05808251f0b5bdaa4b95e78ec83926c981e5a8e502ddf17",
"pubkey": "241fb407a16e1667448ef27584310ab4fd0dfd190eefb7fb2d94054b78210dab",
"created_at": 1709247775,
"kind": 1,
"tags": [],
"content": "Currently writing an implementation of the 1-bit quantization linear layer from this paper. Fresh trained LLM costs go to ~zero if this works at scale. https://arxiv.org/pdf/2402.17764.pdf",
"sig": "dd10d6ef4f22792c652aa50993bac650580e546a280a3178aba659df33b9654a43a3aed1942be02209d2f06aa6d5dc30c99f5df1c23eecfc8133d5341797ecca"
}
