I am actually pretty interested in these Intel Core Ultra SoCs (and the comparable Ryzen SoCs) because I think being able to run a ~7B model on-device could be reasonably useful.
That said, I’m waiting until there’s a Linux distro that really brings the NPUs to the party. I do not trust any OS to do with that resource as it sees fit. I want to know 100% of what it’s doing.
https://mastodon.social/@nixCraft/113074005537132661
Matt May /
npub10t…s9mm4
2024-09-03 20:29:03
Author Public Key
npub10tdxygxx0yadrdfvj29dxdsg3w7pvps32xf7k6dkdvlzqpsjfx2qks9mm4Published at
2024-09-03 20:29:03Event JSON
{
"id": "ee6ba305291dcf15a0254daf1d46adb9bc9f720f02543502cfd20ee457d73ee2",
"pubkey": "7ada6220c6793ad1b52c928ad336088bbc1606115193eb69b66b3e2006124994",
"created_at": 1725395343,
"kind": 1,
"tags": [
[
"proxy",
"https://mstdn.social/@mattmay/113075509263513462",
"web"
],
[
"proxy",
"https://mstdn.social/users/mattmay/statuses/113075509263513462",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mstdn.social/users/mattmay/statuses/113075509263513462",
"pink.momostr"
],
[
"-"
]
],
"content": "I am actually pretty interested in these Intel Core Ultra SoCs (and the comparable Ryzen SoCs) because I think being able to run a ~7B model on-device could be reasonably useful.\n\nThat said, I’m waiting until there’s a Linux distro that really brings the NPUs to the party. I do not trust any OS to do with that resource as it sees fit. I want to know 100% of what it’s doing.\nhttps://mastodon.social/@nixCraft/113074005537132661",
"sig": "11072c6902ada75542215209a0645d58f321ee3289c606cfc5d20cff329c8238a734148ace6bd770e4aa85208de4b8df2bd6e0f56a47550ebc47678a192c7514"
}