Derek Ross on Nostr: Umbrel releases LlamaGPT - A self-hosted, offline, private AI chatbot that can be ...

Umbrel releases LlamaGPT - A self-hosted, offline, private AI chatbot that can be installed on any device, not just Umbrel OS.

quoting
note1c0y…ss92

Introducing LlamaGPT — a self-hosted, offline and private AI chatbot, powered by Llama 2, with absolutely no data leaving your device. 🔐

Yes, an entire LLM. ✨

Your Umbrel Home, Raspberry Pi (8GB) Umbrel, or custom umbrelOS server can run it with just 5GB of RAM!

Word generation benchmarks:

Umbrel Home: ~3 words/sec
Raspberry Pi (8GB RAM): ~1 word/sec

→ Watch the demo: https://youtu.be/iu3_1a8SzeA

→ Install on umbrelOS: https://apps.umbrel.com/app/llama-gpt

→ GitHub: https://github.com/getumbrel/llama-gpt