jb55 on Nostr: Apple’s on-device ai model is 3B params, 2 to 4bit quantization: “On this ...
Apple’s on-device ai model is 3B params, 2 to 4bit quantization:
“On this benchmark, our on-device model, with ~3B parameters, outperforms larger models including Phi-3-mini, Mistral-7B, and Gemma-7B. Our server model compares favorably to DBRX-Instruct, Mixtral-8x22B, and GPT-3.5-Turbo while being highly efficient.”
Interesting! This was the size of model I was considering for damus mobile. Looks like I can just use apple intelligence apis instead 🤔 . These small local models are pretty good at summarization, which I’m guessing why they showcased that a lot in notifications, mail, imessage, etc.
https://machinelearning.apple.com/research/introducing-apple-foundation-modelsPublished at
2024-06-11 18:20:45Event JSON
{
"id": "3a740331fbdcf5185cd7c0d08e437bbf2b562f24c94a0ae62d92f7859d65f803",
"pubkey": "32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245",
"created_at": 1718130045,
"kind": 1,
"tags": [
[
"r",
"https://machinelearning.apple.com/research/introducing-apple-foundation-models"
]
],
"content": "Apple’s on-device ai model is 3B params, 2 to 4bit quantization:\n\n“On this benchmark, our on-device model, with ~3B parameters, outperforms larger models including Phi-3-mini, Mistral-7B, and Gemma-7B. Our server model compares favorably to DBRX-Instruct, Mixtral-8x22B, and GPT-3.5-Turbo while being highly efficient.”\n\nInteresting! This was the size of model I was considering for damus mobile. Looks like I can just use apple intelligence apis instead 🤔 . These small local models are pretty good at summarization, which I’m guessing why they showcased that a lot in notifications, mail, imessage, etc.\n\nhttps://machinelearning.apple.com/research/introducing-apple-foundation-models",
"sig": "c99f3a96a444ba4c1c4ec8546aae28926c1da2963d45efd915e3ef5b477e951c1166ac5a3b9ffb20ad3f3c430cd1f9ba6a6b8e060981accdab2096f4b25cf885"
}
