How to run model

[BioStarr]

Hello I would like to ask just how to run model with as less code as possible.

I have laptop with intel CPU and no GPU

I have written following code but once i run it the entire machine just freezes

Ram is 16gb
CPU is 8 Core
Os is windows 11

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

tokenizer = AutoTokenizer.from_pretrained("gorilla-llm/gorilla-mpt-7b-hf-v0", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("gorilla-llm/gorilla-mpt-7b-hf-v0", trust_remote_code=True)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, trust_remote_code=True,device=-1)

question = "Biggest city in the world?"

response = pipe(question, max_length=50, num_return_sequences=1, do_sample=False)

print("Question:", question)
print("Answer:", response[0]['generated_text'])

[ShishirPatil]

Hey @BioStarr, I'd recommend you try out our colabs to get started:

To run it locally like you want to, we'd recommend you try the code stub from here: https://github.com/ShishirPatil/gorilla/tree/main/inference#1-inference-using-cli