Matty (nprofile…x8tf)
you're not gonna get much done with the p4000's 8gb unfortunately, the best you can do is some light image generation. if you want to host a llm there really isn't a point in running the generalist 7-13b models that could fit in 8gb (± some offloaded layers to system ram) but I've heard that some ""coding assistant"" models are alright
Xeraser /
npub1aj…hpmr7
2024-12-15 23:55:26
in reply to nevent1q…w4rc
Author Public Key
npub1ajv2lxlnghg95sze62cxs77zm7u6gg8sh25qy7cmw79wnjhruwzqshpmr7Published at
2024-12-15 23:55:26Event JSON
{
"id": "d2886563448700032bc651747d4932431a612f2c941c0831ab0b46a5899b9d79",
"pubkey": "ec98af9bf345d05a4059d2b0687bc2dfb9a420f0baa8027b1b778ae9cae3e384",
"created_at": 1734306926,
"kind": 1,
"tags": [
[
"p",
"8b347916be2cb3ab9687c9eb78a8d05224c045bce5b416bdd50169965eb0f45c",
"wss://relay.mostr.pub"
],
[
"e",
"d7c185bd24103984416195327b79684fcebe50d4b63bdbb0878bbc02a31c29a7",
"wss://relay.mostr.pub",
"reply"
],
[
"proxy",
"https://varishangout.net/objects/67c3a233-0535-4a03-85f6-be26c7b6b39a",
"activitypub"
]
],
"content": "nostr:nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpq3v68j9479je6h958e84h32xs2gjvq3duuk6pd0w4q95evh4s73wq2tx8tf \nyou're not gonna get much done with the p4000's 8gb unfortunately, the best you can do is some light image generation. if you want to host a llm there really isn't a point in running the generalist 7-13b models that could fit in 8gb (± some offloaded layers to system ram) but I've heard that some \"\"coding assistant\"\" models are alright",
"sig": "99f4af6bf3f4ac1dcd16f1436bd3f819e70e14d3f492b2647b0863e47713c2c0b19150c93728da18161c2d63041d35c5fa108c2f0e669731e3ae93b9143905ab"
}