Oh good grief. 😂 Then yeah, it is all explained by your RAM speed, you aren't going to get faster on that machine.
You can do a Mixture of Experts Model, like llama 4, to get the number of active parameters down, but it is still going to be slow and performance will be worse than a full model like llama3.3.
Fast memory is everything for AI workloads.
Daniel Wigton
npub1w4…q30zp
2025-05-24 07:45:08
in reply to nevent1q…f0p3
Author Public Key
npub1w4jkwspqn9svwnlrw0nfg0u2yx4cj6yfmp53ya4xp7r24k7gly4qaq30zpPublished at
2025-05-24 07:45:08Event JSON
{
"id": "4e66d29cd02d9b6710d0a2ff90e07b1db6d30ff8d74a9430fc434dd0667f3e87",
"pubkey": "75656740209960c74fe373e6943f8a21ab896889d8691276a60f86aadbc8f92a",
"created_at": 1748072708,
"kind": 1,
"tags": [
[
"e",
"d090889dfc19e313ca6d93f90197e1095105ea76fa4c8235e7cb8597f9953694",
"",
"root"
],
[
"e",
"0ac47e03bbafa8dbed8e5396d9251e065cfa095c9ba5b89125945e5ff78cd441"
],
[
"e",
"8d3112e539b82ddcaf8b72b7d42e3f31c86bbe80bee447de810367fdd0300e62",
"",
"reply"
],
[
"p",
"036533caa872376946d4e4fdea4c1a0441eda38ca2d9d9417bb36006cbaabf58"
],
[
"p",
"75656740209960c74fe373e6943f8a21ab896889d8691276a60f86aadbc8f92a"
]
],
"content": "Oh good grief. 😂 Then yeah, it is all explained by your RAM speed, you aren't going to get faster on that machine.\n\nYou can do a Mixture of Experts Model, like llama 4, to get the number of active parameters down, but it is still going to be slow and performance will be worse than a full model like llama3.3.\n\nFast memory is everything for AI workloads.",
"sig": "9b5b56ec3a944042fdeb9e30385e257d2127f03e74387763881c9af1daf20fd3aaccf529e29648dcd2189e70842a58ad9a40cfd61848c15fb5938bd578e37dc6"
}