Simon Willison on Nostr: one of the many reasons I'm so interested in smaller models that can run directly on ...

one of the many reasons I'm so interested in smaller models that can run directly on a laptop or phone - like Mistral 7B - is that the inference energy costs are then reduced to the cost of running one of those devices - and the one-time training cost gets shared between everyone using that model