Experimental Standalone and Llama 2 64bit Portable Binary
Pre-release
Pre-release
trholding
released this
27 Jul 05:25
·
536 commits
to master
since this release
This is a experimental release compiled with the following features:
- Aarch64 support (slow, emulated with blink, ARM64 Mac, 64bit Raspi, ARM 64 bit machines)
- Baremetal Support (boot but not fully functiona, can't open model)
- Standalone (Embedded model)
Due to standalone/baremetal support, this build will ignore cli arguments.
Just copy and run run.com in a cli on any 64 Bit OS
*BSD/Linux Example:
$ ./run.com
Windows Example:
C:> run.com
Initial bootup is slower as model will be uncompressed first.
x86_64 performance is a bit higher with cosmocc non openmp builds.
Test *baremetal boot like this:
qemu-system-x86_64 -serial stdio -hda run.com
*baremetal boots but not yet fully functional as model file can't be opened without a fix in toolchain.
Built with:
cosmocc -Ofast -funsafe-math-optimizations -ffast-math \
-D COSMO_BLINK -D COSMO_METAL -D COSMO_ZIP -o run.com run.c -lm
Includes the following model:
https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin