Why Nostr? What is Njump?
2023-11-18 15:50:44
in reply to

Zeugs on Nostr: npub14ujlk…p4dcp do you you need about a few hundred gb of GPU ram if you want to ...

do you you need about a few hundred gb of GPU ram if you want to use a single instance of an state of the art LLM Falcon 180b needs 720GB graphics memory. That's expensive. Even if you run it at fp16 it's 360 GB. And that's just a single instance(I have no idea how much clients you can supply with a single instance). You definitely need to talk to sales if you want some instances of cloud hardware that can handle such big lllm's.
Author Public Key
npub1f996px72tylw877aalhysuk8nvtm9srpa2a6xp6n6f3yvga76dzsv2309f