Check out our HuggingFace space for a live demo!
// Asynchronous loading & caching with IndexedDB
let model = await Model.load(AvailableModels.WHISPER_TINY, Quantization.Q8, (p: number) => setProgress(p))
let result = await model.run({ input });
Rust crate & CLI coming soon...
We want a toolkit for developers to make integrating performant AI functionality into existing production applications easy. The following principles will help us accomplish this:
- Inference only
- WebGPU/CPU only
- First class quantization support
- Lazy computation
- Inplace by default
- Whisper
- Coming soon...