Run nix run github:jim3692/koboldcpp-flake
It downloads Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf using huggingface-cli, if it doesn't already exist in ~/.cache/huggingface, and then starts KoboldCpp in Vulkan mode.
-
Notifications
You must be signed in to change notification settings - Fork 0
jim3692/koboldcpp-flake
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|