Skip to content

Experimental Standalone and Llama 2 64bit Portable Binary

Pre-release
Pre-release
Compare
Choose a tag to compare
@trholding trholding released this 27 Jul 05:25
· 536 commits to master since this release
7b31221

This is a experimental release compiled with the following features:

  • Aarch64 support (slow, emulated with blink, ARM64 Mac, 64bit Raspi, ARM 64 bit machines)
  • Baremetal Support (boot but not fully functiona, can't open model)
  • Standalone (Embedded model)

Due to standalone/baremetal support, this build will ignore cli arguments.

Just copy and run run.com in a cli on any 64 Bit OS

*BSD/Linux Example:

$ ./run.com

Windows Example:

C:> run.com

Initial bootup is slower as model will be uncompressed first.

x86_64 performance is a bit higher with cosmocc non openmp builds.

Test *baremetal boot like this:

qemu-system-x86_64 -serial stdio -hda run.com

*baremetal boots but not yet fully functional as model file can't be opened without a fix in toolchain.

Built with:

cosmocc -Ofast -funsafe-math-optimizations -ffast-math  \ 
-D COSMO_BLINK -D COSMO_METAL -D COSMO_ZIP -o run.com run.c -lm

Includes the following model:

https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin