You can use Ollama for inferring some playground models. It can run on both CPU and GPU.
Python is required.
Learn how to create the dataset (in ChatML format or completion format, for example).
For fine tuning, if you have Nvidia GPU(s), try using Unsloth.
As for me, I'm using my MacBook with the MLX framework (built for Apple Silicon) to fine-tune and infer my LLMs in one place. It's very easy to start with self-hosted solutions.
gooGof
npub1uu…ue0l3
2024-04-22 06:26:05
in reply to nevent1q…7dv2
Author Public Key
npub1uuy6vl9tr8jru4l0qanmuzgs7qkk8kn46xs84fk3kjgvypcwnekqwue0l3Published at
2024-04-22 06:26:05Event JSON
{
"id": "75464a2817f06c520f39fd532bfee4cb0f3bc3f27129b141bc96f1f4a252e0d1",
"pubkey": "e709a67cab19e43e57ef0767be0910f02d63da75d1a07aa6d1b490c2070e9e6c",
"created_at": 1713767165,
"kind": 1,
"tags": [
[
"e",
"00018746f726e1ffe36b5596d6fd38662a80a3eb01e618fb38eff133a15a87f1",
"",
"root"
],
[
"p",
"08c9d8c533064607a7f274961f7b375f3d2fc24ba11b77333c76c0d8377ab1f3"
]
],
"content": "You can use Ollama for inferring some playground models. It can run on both CPU and GPU.\n\nPython is required.\n\nLearn how to create the dataset (in ChatML format or completion format, for example).\n\nFor fine tuning, if you have Nvidia GPU(s), try using Unsloth.\n\nAs for me, I'm using my MacBook with the MLX framework (built for Apple Silicon) to fine-tune and infer my LLMs in one place. It's very easy to start with self-hosted solutions.",
"sig": "75d39714f14f2313107d26409229ca180ad9c6894cedb210226ad8f69d5ea6cc7381d9778c91494eda3ad8870c8c073b0d9a8c06eb3c5578155e3fe9e6e83183"
}