🥥Matthew Martin🥥☑ on Nostr: It works but like said elsewhere, painfully slow, you'll be limited to the smallest ...

It works but like said elsewhere, painfully slow, you'll be limited to the smallest models (some are surprisingly good for their size, but not good compared to frontier models). So dialog from an NPC, or maybe a simple classification problem that would otherwise be tricky to code, a grammar checker. But these models that can run locally aren't substitutes for ChatGPT/Claude. I'm waiting for consumer grade hardware that I can afford and can run 70B+ models fast.