My new AI Tower computer has a NVIDEA 12GB 3060 GPU and 32GB of RAM. It runs #LLM 32b (ish) models with #Ollama nicely but it won’t run 70b models without more RAM.
Could I get opinions on the benefits of 70b models vs 32b models. My long term purpose is to research LLM and #AI usage for #edtech, especially #evaluativeai for #Moodle
Marcus Green /
npub1h4…8nmje
2025-02-21 09:20:54
Author Public Key
npub1h4d544vfux5yvp20kmauz86gruez5p39srm2m6uk7xf5p5hm42yqy8nmjeSeen on
wss://relay.mostr.pubPublished at
2025-02-21 09:20:54Event JSON
{
"id": "a13bb012d74559aae36f90e00772894ade933e1a0303673998e4a2ff6985cdd5",
"pubkey": "bd5b4ad589e1a846054fb6fbc11f481f322a062580f6adeb96f19340d2fbaa88",
"created_at": 1740129654,
"kind": 1,
"tags": [
[
"t",
"llm"
],
[
"t",
"ollama"
],
[
"t",
"ai"
],
[
"t",
"edtech"
],
[
"t",
"evaluativeai"
],
[
"t",
"moodle"
],
[
"proxy",
"https://fosstodon.org/users/marcusgreen/statuses/114041137047330776",
"activitypub"
]
],
"content": "My new AI Tower computer has a NVIDEA 12GB 3060 GPU and 32GB of RAM. It runs #LLM 32b (ish) models with #Ollama nicely but it won’t run 70b models without more RAM. \n\nCould I get opinions on the benefits of 70b models vs 32b models. My long term purpose is to research LLM and #AI usage for #edtech, especially #evaluativeai for #Moodle",
"sig": "d42d73ce87110c31726877ef094d496bed6084e6b455a1932c067b5b1fe1fc0436b0506a9e11dff2ef8d177c19e8624fb2c8d0783d70ec7edef7a096c79c551e"
}